From 8953883b3f9893a5f8934ed0a89585cf7bd4d474 Mon Sep 17 00:00:00 2001 From: Gustavo Medori Date: Sun, 12 Apr 2026 22:22:49 -0600 Subject: [PATCH 01/12] Add initial draft for nnnn-tag-based-test-execution-filtering.md --- ...nnnn-tag-based-test-execution-filtering.md | 111 ++++++++++++++++++ 1 file changed, 111 insertions(+) create mode 100644 proposals/testing/nnnn-tag-based-test-execution-filtering.md diff --git a/proposals/testing/nnnn-tag-based-test-execution-filtering.md b/proposals/testing/nnnn-tag-based-test-execution-filtering.md new file mode 100644 index 0000000000..cb0d71e21a --- /dev/null +++ b/proposals/testing/nnnn-tag-based-test-execution-filtering.md @@ -0,0 +1,111 @@ +# Tag-based Test Execution Filtering + +- Proposal: [ST-NNNN](nnnn-tag-based-test-execution-filtering.md) +- Authors: [Gustavo Medori](https://github.com/gmedori) +- Review Manager: TBD +- Status: **Awaiting review** +- Implementation: [swiftlang/swift-testing#1531](https://github.com/swiftlang/swift-testing/pull/1531) +- Review: ([pitch (not yet live)](https://forums.swift.org/...)) + +## Introduction + +Swift Testing currently provides the ability to annotate tests and suites with arbitrarily named tags. It also allows you to selectively run specific tests by matching them against a regex using the `--filter` and `--skip` command line options. I propose we join these two capabilities to allow for filtering or skipping tests based on any associated tags. + +## Motivation + +Tests come in all shapes and sizes, from narrowly-scoped unit tests, to integration tests that spin up live databases, to UI tests that can take a very long time to run. Depending on your needs, you might want to control which tests are run and when, rather than running your entire test suite every time with `swift test`. Swift Testing acknowledges this need by the existence of the `--filter` and `--skip` command line options, but these options fall short in certain scenarios. + +Consider an iOS project where your UI code is spread across several targets (e.g. one target per "feature"). Suppose further that each of these targets has an associated test target with its own suite of tests. It could look something like this: + +``` +FoodTruck/ +├── Package.swift +├── Sources/ +│   ├── FoodDetailFeature/ +│   ├── FoodListFeature/ +│   ├── FoodTruck/ +│   └── RootFeature/ +└── Tests/ + ├── FoodDetailFeatureTests/ + ├── FoodListFeatureTests/ + ├── FoodTruckTests/ + └── RootFeatureTests/ +``` + +During local development, you may want to skip all UI tests across all your test packages. With current tooling, this isn't possible unless you have a consistent naming scheme for your UI tests across all your packages that don't overlap with any other tests—a tall order in larger codebases. For this purpose, it is clear that we need a better, user-defined, way of grouping tests together outside of the test graph. + +## Proposed Solution + +I propose we introduce a special syntax to the `--filter` and `--skip` command line options. When you want to filter or skip tests by providing a tag, you will prefix the argument with `tag:`. + +## Detailed Design + +### Basic Usage + + As they currently exist, the `--filter` and `--skip` command line options accept regular expressions as arguments. Reiterating the above, I propose enhancing these arguments to accept a special case: an exact tag name prefixed by `tag:`. For example: + +``` +swift test --skip tag:uiTest +``` + +In this example, `uiTest` must be the _exact_ name of the tag. Tags _will not_ match fuzzily or by regular expression. This treatment would be applied to both the `--filter` and `--skip` options. + +As with regular usages of `--filter`/`--skip`, you can supply the option multiple times to filter/skip tests which match _any_ of the tags: + +``` +swift test --skip tag:uiTest --skip tag:integrationTest +``` + +It is not currently possible to create a filter/skip based on _all_ tags specified, as opposed to _any_ of the tags specified. In other words, supplying multiple `--filter`/`--skip` options is an **or** operation, not an **and** operation. + +### Handling Raw Identifiers + + As of [SE-0451](https://github.com/swiftlang/swift-evolution/blob/main/proposals/0451-escaped-identifiers.md), Swift has raw identifiers which means the following is valid Swift: + +```swift +@Test func `tag:uiTest`() { /* ... */ } +``` + +To continue to allow matching for such function names, we will also allow the colon character to be escaped such that the entire argument will be treated as a single regular expression: + +``` +swift test --skip 'tag\:uiTest' +``` + +The example above would behave as though the string `tag:uiTest` were passed as a regular expression, omitting the escaping backslash in the final regular expression. + +> **Note**: Most shells treat the backslash character `\` as a special character used for escaping. In order for the application to receive it, the argument needs to either be wrapped in quotes like `'tag\:uiTest'`, or the backslash itself needs to be escaped, `tag\\:uiTest`. + +## Source Compatibility + +The change is an implementation detail and makes no changes to source compatibility. + +## Integration with Supporting Tools + +If a codebase has test functions or suites that contain the string `tag:`, then any filtering/skipping arguments that begin with `tag:` behave differently with this proposal. They are treated as tags rather than regular expressions. + +It's worth noting that string `tag:` can only appear in symbol names when using raw identifiers as described above. I feel that this scenario is relatively rare and that this improvement is worth the edge case. + +## Future Directions + +N/A + +## Alternatives Considered + +An alternate path is to create a separate command line option for filtering and skipping by tags. For example: + +``` +swift test --skipTag uiTest +``` + +While this approach may seem simpler, it has a few disadvantages: + +1. We would need to make changes to the ABI between SwiftPM and Swift Testing. +2. A separate flag means a separate line in the `--help` text, making it easier to miss that filtering/skipping based on tags is possible. +3. The implementation would be more complex, requiring changes across both [swiftlang/swift-testing](https://github.com/swiftlang/swift-testing) and [swiftlang/swift-package-manager](https://github.com/swiftlang/swift-package-manager). More code = more bugs, on average. + +The main advantage is that we wouldn't need to handle escaping the colon character, and that didn't seem like a big enough benefit to warrant the disadvantages. + +## Acknowledgments + +TBD From 61e2f4b2caf246a318cdafa0f1b20d3aa4e1f553 Mon Sep 17 00:00:00 2001 From: Gustavo Medori Date: Mon, 13 Apr 2026 16:01:52 -0600 Subject: [PATCH 02/12] Add ideas for future directions we could take skipping/filtering --- .../testing/nnnn-tag-based-test-execution-filtering.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/proposals/testing/nnnn-tag-based-test-execution-filtering.md b/proposals/testing/nnnn-tag-based-test-execution-filtering.md index cb0d71e21a..74c7f0db00 100644 --- a/proposals/testing/nnnn-tag-based-test-execution-filtering.md +++ b/proposals/testing/nnnn-tag-based-test-execution-filtering.md @@ -88,7 +88,11 @@ It's worth noting that string `tag:` can only appear in symbol names when using ## Future Directions -N/A +Filtering based on tags is quite broad and general purpose. Because you can define any tag to stick on any test or suite, and because tags exists orthogonally to the test graph, you can arbitrarily include/skip any test based solely on the semantics of your tags. However, this change does raise the question of what _else_ we could filter/skip on and how we can be more expressive about it. + +For example, you may wish to filter/skip tests based on protocol conformance and/or inheritance. A suite's ancestor types can be a useful, and perhaps more natural, signal indicating whether it should run in a given context or not because the ancestor types carry with them behaviors and contracts that have powerful semantic meaning. In the future, we may seek to expand the prefix operators we allow beyong just `tag:`. + +Additionally, internally, the test suite already supports arbitrary boolean groupings of test filters. It may not be unreasonable to attempt to expose that on the CLI. ## Alternatives Considered @@ -98,7 +102,7 @@ An alternate path is to create a separate command line option for filtering and swift test --skipTag uiTest ``` -While this approach may seem simpler, it has a few disadvantages: +This approach may seem simpler, and indeed one important advantage is that we would forgo the need to have a well-defined syntax for filtering/skipping if we decide to enhance it further. However, it has a few disadvantages: 1. We would need to make changes to the ABI between SwiftPM and Swift Testing. 2. A separate flag means a separate line in the `--help` text, making it easier to miss that filtering/skipping based on tags is possible. From d80d77b71f7f63192f1fc722c96821abdc1c707f Mon Sep 17 00:00:00 2001 From: Gustavo Medori Date: Mon, 13 Apr 2026 16:03:10 -0600 Subject: [PATCH 03/12] Fix some markdown formatting --- proposals/testing/nnnn-tag-based-test-execution-filtering.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/testing/nnnn-tag-based-test-execution-filtering.md b/proposals/testing/nnnn-tag-based-test-execution-filtering.md index 74c7f0db00..204d98f6ef 100644 --- a/proposals/testing/nnnn-tag-based-test-execution-filtering.md +++ b/proposals/testing/nnnn-tag-based-test-execution-filtering.md @@ -68,13 +68,14 @@ It is not currently possible to create a filter/skip based on _all_ tags specifi To continue to allow matching for such function names, we will also allow the colon character to be escaped such that the entire argument will be treated as a single regular expression: -``` +```sh swift test --skip 'tag\:uiTest' ``` The example above would behave as though the string `tag:uiTest` were passed as a regular expression, omitting the escaping backslash in the final regular expression. -> **Note**: Most shells treat the backslash character `\` as a special character used for escaping. In order for the application to receive it, the argument needs to either be wrapped in quotes like `'tag\:uiTest'`, or the backslash itself needs to be escaped, `tag\\:uiTest`. +> [!Note] +> Most shells treat the backslash character `\` as a special character used for escaping. In order for the application to receive it, the argument needs to either be wrapped in quotes like `'tag\:uiTest'`, or the backslash itself needs to be escaped, `tag\\:uiTest`. ## Source Compatibility From 90054874fa579ab3c39d512f624b24ca13a9bc97 Mon Sep 17 00:00:00 2001 From: Gustavo Medori Date: Mon, 13 Apr 2026 16:05:23 -0600 Subject: [PATCH 04/12] Reorganized some stuff around source compatibility and added more to the discussion of raw identifiers --- ...nnnn-tag-based-test-execution-filtering.md | 24 ++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/proposals/testing/nnnn-tag-based-test-execution-filtering.md b/proposals/testing/nnnn-tag-based-test-execution-filtering.md index 204d98f6ef..cf052782e4 100644 --- a/proposals/testing/nnnn-tag-based-test-execution-filtering.md +++ b/proposals/testing/nnnn-tag-based-test-execution-filtering.md @@ -77,16 +77,34 @@ The example above would behave as though the string `tag:uiTest` were passed as > [!Note] > Most shells treat the backslash character `\` as a special character used for escaping. In order for the application to receive it, the argument needs to either be wrapped in quotes like `'tag\:uiTest'`, or the backslash itself needs to be escaped, `tag\\:uiTest`. -## Source Compatibility -The change is an implementation detail and makes no changes to source compatibility. +The converse here also applies. It is possible to apply a tag that uses a raw identifier as its name. For example: -## Integration with Supporting Tools +```swift +extension Tag { + @Tag static var `some tag with spaces`: Self +} + +@Test(.tags(.`some tag with spaces`)) +func myTest() { /* ... */ } +``` + +In this scenario, you would wrap the argument in single quotes and the entirity of the text supplied after the `tag:` prefix would be interpreted the name of a single tag. So for a tag named `some tag with spaces`, you could filter for it as follows: + +```sh +swift test --filter 'tag:some tag with spaces' +``` + +## Source Compatibility If a codebase has test functions or suites that contain the string `tag:`, then any filtering/skipping arguments that begin with `tag:` behave differently with this proposal. They are treated as tags rather than regular expressions. It's worth noting that string `tag:` can only appear in symbol names when using raw identifiers as described above. I feel that this scenario is relatively rare and that this improvement is worth the edge case. +## Integration with Supporting Tools + +This introduces a new mechanism that can be used by any existing tools to filter/skip based on tags, and has the same backwards compatibility concerns associated with the source compatibility section above. That is, if a tool that calls `swift test` is filtering/skipping for a test with a name containing `tag:`, this will cause unexpected behavior. Otherwise, this change is additive. + ## Future Directions Filtering based on tags is quite broad and general purpose. Because you can define any tag to stick on any test or suite, and because tags exists orthogonally to the test graph, you can arbitrarily include/skip any test based solely on the semantics of your tags. However, this change does raise the question of what _else_ we could filter/skip on and how we can be more expressive about it. From 15ecec94b448d50cb168176897b876cf1a8dc378 Mon Sep 17 00:00:00 2001 From: Gustavo Medori Date: Mon, 4 May 2026 15:09:08 -0600 Subject: [PATCH 05/12] Adjust proposal with new `id:` prefix Discussion here: https://forums.swift.org/t/pitch-tag-based-test-execution-filtering/86001/11 --- .../nnnn-tag-based-test-execution-filtering.md | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/proposals/testing/nnnn-tag-based-test-execution-filtering.md b/proposals/testing/nnnn-tag-based-test-execution-filtering.md index cf052782e4..52126f5cef 100644 --- a/proposals/testing/nnnn-tag-based-test-execution-filtering.md +++ b/proposals/testing/nnnn-tag-based-test-execution-filtering.md @@ -48,9 +48,7 @@ I propose we introduce a special syntax to the `--filter` and `--skip` command l swift test --skip tag:uiTest ``` -In this example, `uiTest` must be the _exact_ name of the tag. Tags _will not_ match fuzzily or by regular expression. This treatment would be applied to both the `--filter` and `--skip` options. - -As with regular usages of `--filter`/`--skip`, you can supply the option multiple times to filter/skip tests which match _any_ of the tags: +In this example, `uiTest` is a regular expression that matches the tags that you want to filter/skip on. As with regular usages of `--filter`/`--skip`, you can supply the option multiple times to filter/skip tests which match _any_ of the tags: ``` swift test --skip tag:uiTest --skip tag:integrationTest @@ -66,19 +64,17 @@ It is not currently possible to create a filter/skip based on _all_ tags specifi @Test func `tag:uiTest`() { /* ... */ } ``` -To continue to allow matching for such function names, we will also allow the colon character to be escaped such that the entire argument will be treated as a single regular expression: +To continue to allow matching for such function names, I propose introducing a _separate_ prefix called `id:` which behaves much like the `tag:` prefix in that everything that follows it is a regular expression. It's job, however, is to disambiguate and allow the user a mechanism to explicitly say "match on test symbol names please." ```sh -swift test --skip 'tag\:uiTest' +swift test --skip 'id:tag:uiTest' ``` -The example above would behave as though the string `tag:uiTest` were passed as a regular expression, omitting the escaping backslash in the final regular expression. - -> [!Note] -> Most shells treat the backslash character `\` as a special character used for escaping. In order for the application to receive it, the argument needs to either be wrapped in quotes like `'tag\:uiTest'`, or the backslash itself needs to be escaped, `tag\\:uiTest`. +The `id:` prefix doesn't introduce any new behavior. In fact, its behavior is the entirety of what Swift Testing supports today. However, we wanted a flexible way to disambiguate raw identifiers that also left the door open for other filtering/skipping mechanisms in the future. +It will still be possible to omit the prefix entirely. So long as nothing prefix-shaped exists (i.e. roughly: non-colon characters followed by a colon), we will assume matching by `id:`. -The converse here also applies. It is possible to apply a tag that uses a raw identifier as its name. For example: +With respect to raw identifiers, the converse here also applies. It is possible to apply a tag that uses a raw identifier as its name. For example: ```swift extension Tag { From 0baa9230ddfbd4f0687ffb6afa1b1ad175e309a4 Mon Sep 17 00:00:00 2001 From: Gustavo Medori Date: Mon, 4 May 2026 15:24:18 -0600 Subject: [PATCH 06/12] Tweak wording on when we assume matching by `id:` --- proposals/testing/nnnn-tag-based-test-execution-filtering.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/testing/nnnn-tag-based-test-execution-filtering.md b/proposals/testing/nnnn-tag-based-test-execution-filtering.md index 52126f5cef..841cb1cc1e 100644 --- a/proposals/testing/nnnn-tag-based-test-execution-filtering.md +++ b/proposals/testing/nnnn-tag-based-test-execution-filtering.md @@ -72,7 +72,7 @@ swift test --skip 'id:tag:uiTest' The `id:` prefix doesn't introduce any new behavior. In fact, its behavior is the entirety of what Swift Testing supports today. However, we wanted a flexible way to disambiguate raw identifiers that also left the door open for other filtering/skipping mechanisms in the future. -It will still be possible to omit the prefix entirely. So long as nothing prefix-shaped exists (i.e. roughly: non-colon characters followed by a colon), we will assume matching by `id:`. +It will still be possible to omit the prefix entirely. So long as none of the known prefixes are present, we will assume matching by `id:`. With respect to raw identifiers, the converse here also applies. It is possible to apply a tag that uses a raw identifier as its name. For example: From a2b375a4afe2f7894c176970d573a5176bf3768d Mon Sep 17 00:00:00 2001 From: Gustavo Medori Date: Tue, 19 May 2026 14:49:26 -0600 Subject: [PATCH 07/12] Add a section on VS Code motivation and how we handle surrounding backticks supplied by the user in a filter --- .../nnnn-tag-based-test-execution-filtering.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/proposals/testing/nnnn-tag-based-test-execution-filtering.md b/proposals/testing/nnnn-tag-based-test-execution-filtering.md index 841cb1cc1e..d3576648d3 100644 --- a/proposals/testing/nnnn-tag-based-test-execution-filtering.md +++ b/proposals/testing/nnnn-tag-based-test-execution-filtering.md @@ -34,6 +34,8 @@ FoodTruck/ During local development, you may want to skip all UI tests across all your test packages. With current tooling, this isn't possible unless you have a consistent naming scheme for your UI tests across all your packages that don't overlap with any other tests—a tall order in larger codebases. For this purpose, it is clear that we need a better, user-defined, way of grouping tests together outside of the test graph. +Additionally, it's worth noting that existing tools that support filtering by tags (like the Visual Studio Code plugin) do so by using the SourceKit index to search for tags and then collecting the test IDs that match those tags into a [giant regular expression](https://github.com/swiftlang/vscode-swift/blob/f56817494c1ea989dbeec894896be98dd8e25c8a/src/TestExplorer/TestRunArguments.ts#L99-L118). Adding a native ability to filter by tag would simplify this implementation (and that of any other tools hoping to implement the same). + ## Proposed Solution I propose we introduce a special syntax to the `--filter` and `--skip` command line options. When you want to filter or skip tests by providing a tag, you will prefix the argument with `tag:`. @@ -91,6 +93,18 @@ In this scenario, you would wrap the argument in single quotes and the entirity swift test --filter 'tag:some tag with spaces' ``` +It's reasonable to expect some developers to attempt that filter by including the backticks used to delimit the symbol name for a raw identifier like so: + +```sh +swift test --filter 'tag:`some tag with spaces`' # INVALID: This wouldn't match the symbol. +``` + +Backticks are not part of the symbol name of a raw identifier, so this filter wouldn't match anything. I believe it's reasonable to suggest that if some filter the user provides is surrounded by backticks (i.e. more preciesly, it matches the regex ```/^`[^`]*`$/```), then we can be reasonably sure the user means a raw identifier, and we should supply an error message and strip them for the user: + +``` +Backticks aren't a valid part of a Swift symbol. Replacing '`some tag with spaces`' with 'some tag with spaces'. +``` + ## Source Compatibility If a codebase has test functions or suites that contain the string `tag:`, then any filtering/skipping arguments that begin with `tag:` behave differently with this proposal. They are treated as tags rather than regular expressions. From de8386ef6814196b876b07b2befe76e22b67b044 Mon Sep 17 00:00:00 2001 From: Gustavo Medori Date: Wed, 20 May 2026 13:28:40 -0600 Subject: [PATCH 08/12] Update pitch with proposal number and pitch link --- ...iltering.md => 0025-tag-based-test-execution-filtering.md} | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename proposals/testing/{nnnn-tag-based-test-execution-filtering.md => 0025-tag-based-test-execution-filtering.md} (98%) diff --git a/proposals/testing/nnnn-tag-based-test-execution-filtering.md b/proposals/testing/0025-tag-based-test-execution-filtering.md similarity index 98% rename from proposals/testing/nnnn-tag-based-test-execution-filtering.md rename to proposals/testing/0025-tag-based-test-execution-filtering.md index d3576648d3..06fc525ab5 100644 --- a/proposals/testing/nnnn-tag-based-test-execution-filtering.md +++ b/proposals/testing/0025-tag-based-test-execution-filtering.md @@ -1,11 +1,11 @@ # Tag-based Test Execution Filtering -- Proposal: [ST-NNNN](nnnn-tag-based-test-execution-filtering.md) +- Proposal: [ST-0025](0025-tag-based-test-execution-filtering.md) - Authors: [Gustavo Medori](https://github.com/gmedori) - Review Manager: TBD - Status: **Awaiting review** - Implementation: [swiftlang/swift-testing#1531](https://github.com/swiftlang/swift-testing/pull/1531) -- Review: ([pitch (not yet live)](https://forums.swift.org/...)) +- Review: [pitch](https://forums.swift.org/t/pitch-tag-based-test-execution-filtering/86001) ## Introduction From 3873c0f993eff88b94eea90f8e800f898bbabba0 Mon Sep 17 00:00:00 2001 From: Paul LeMarquand Date: Thu, 21 May 2026 09:12:00 -0400 Subject: [PATCH 09/12] Move ST-0025 to Review --- .../0025-tag-based-test-execution-filtering.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/proposals/testing/0025-tag-based-test-execution-filtering.md b/proposals/testing/0025-tag-based-test-execution-filtering.md index 06fc525ab5..30d5f3dfaf 100644 --- a/proposals/testing/0025-tag-based-test-execution-filtering.md +++ b/proposals/testing/0025-tag-based-test-execution-filtering.md @@ -2,8 +2,8 @@ - Proposal: [ST-0025](0025-tag-based-test-execution-filtering.md) - Authors: [Gustavo Medori](https://github.com/gmedori) -- Review Manager: TBD -- Status: **Awaiting review** +* Review Manager: [Paul LeMarquand](https://github.com/plemarquand) +* Status: **Active Review (May 21 - June 4, 2026)** - Implementation: [swiftlang/swift-testing#1531](https://github.com/swiftlang/swift-testing/pull/1531) - Review: [pitch](https://forums.swift.org/t/pitch-tag-based-test-execution-filtering/86001) @@ -32,7 +32,7 @@ FoodTruck/ └── RootFeatureTests/ ``` -During local development, you may want to skip all UI tests across all your test packages. With current tooling, this isn't possible unless you have a consistent naming scheme for your UI tests across all your packages that don't overlap with any other tests—a tall order in larger codebases. For this purpose, it is clear that we need a better, user-defined, way of grouping tests together outside of the test graph. +During local development, you may want to skip all UI tests across all your test packages. With current tooling, this isn't possible unless you have a consistent naming scheme for your UI tests across all your packages that doesn't overlap with any other tests — a tall order in larger codebases. For this purpose, it is clear that we need a better, user-defined, way of grouping tests together outside of the test graph. Additionally, it's worth noting that existing tools that support filtering by tags (like the Visual Studio Code plugin) do so by using the SourceKit index to search for tags and then collecting the test IDs that match those tags into a [giant regular expression](https://github.com/swiftlang/vscode-swift/blob/f56817494c1ea989dbeec894896be98dd8e25c8a/src/TestExplorer/TestRunArguments.ts#L99-L118). Adding a native ability to filter by tag would simplify this implementation (and that of any other tools hoping to implement the same). @@ -44,7 +44,7 @@ I propose we introduce a special syntax to the `--filter` and `--skip` command l ### Basic Usage - As they currently exist, the `--filter` and `--skip` command line options accept regular expressions as arguments. Reiterating the above, I propose enhancing these arguments to accept a special case: an exact tag name prefixed by `tag:`. For example: +As they currently exist, the `--filter` and `--skip` command line options accept regular expressions as arguments. Reiterating the above, I propose enhancing these arguments to accept a special case: an exact tag name prefixed by `tag:`. For example: ``` swift test --skip tag:uiTest @@ -60,7 +60,7 @@ It is not currently possible to create a filter/skip based on _all_ tags specifi ### Handling Raw Identifiers - As of [SE-0451](https://github.com/swiftlang/swift-evolution/blob/main/proposals/0451-escaped-identifiers.md), Swift has raw identifiers which means the following is valid Swift: +As of [SE-0451](https://github.com/swiftlang/swift-evolution/blob/main/proposals/0451-escaped-identifiers.md), Swift has raw identifiers which means the following is valid Swift: ```swift @Test func `tag:uiTest`() { /* ... */ } @@ -87,7 +87,7 @@ extension Tag { func myTest() { /* ... */ } ``` -In this scenario, you would wrap the argument in single quotes and the entirity of the text supplied after the `tag:` prefix would be interpreted the name of a single tag. So for a tag named `some tag with spaces`, you could filter for it as follows: +In this scenario, you would wrap the argument in single quotes and the entirety of the text supplied after the `tag:` prefix would be interpreted the name of a single tag. So for a tag named `some tag with spaces`, you could filter for it as follows: ```sh swift test --filter 'tag:some tag with spaces' @@ -99,7 +99,7 @@ It's reasonable to expect some developers to attempt that filter by including th swift test --filter 'tag:`some tag with spaces`' # INVALID: This wouldn't match the symbol. ``` -Backticks are not part of the symbol name of a raw identifier, so this filter wouldn't match anything. I believe it's reasonable to suggest that if some filter the user provides is surrounded by backticks (i.e. more preciesly, it matches the regex ```/^`[^`]*`$/```), then we can be reasonably sure the user means a raw identifier, and we should supply an error message and strip them for the user: +Backticks are not part of the symbol name of a raw identifier, so this filter wouldn't match anything. I believe it's reasonable to suggest that if some filter the user provides is surrounded by backticks (i.e. more precisely, it matches the regex ```/^`[^`]*`$/```), then we can be reasonably sure the user means a raw identifier, and we should supply an error message and strip them for the user: ``` Backticks aren't a valid part of a Swift symbol. Replacing '`some tag with spaces`' with 'some tag with spaces'. @@ -119,7 +119,7 @@ This introduces a new mechanism that can be used by any existing tools to filter Filtering based on tags is quite broad and general purpose. Because you can define any tag to stick on any test or suite, and because tags exists orthogonally to the test graph, you can arbitrarily include/skip any test based solely on the semantics of your tags. However, this change does raise the question of what _else_ we could filter/skip on and how we can be more expressive about it. -For example, you may wish to filter/skip tests based on protocol conformance and/or inheritance. A suite's ancestor types can be a useful, and perhaps more natural, signal indicating whether it should run in a given context or not because the ancestor types carry with them behaviors and contracts that have powerful semantic meaning. In the future, we may seek to expand the prefix operators we allow beyong just `tag:`. +For example, you may wish to filter/skip tests based on protocol conformance and/or inheritance. A suite's ancestor types can be a useful, and perhaps more natural, signal indicating whether it should run in a given context or not because the ancestor types carry with them behaviors and contracts that have powerful semantic meaning. In the future, we may seek to expand the prefix operators we allow beyond just `tag:`. Additionally, internally, the test suite already supports arbitrary boolean groupings of test filters. It may not be unreasonable to attempt to expose that on the CLI. From 31601adaeee013eef95bdef35368bca0f50ef826 Mon Sep 17 00:00:00 2001 From: Paul LeMarquand Date: Thu, 21 May 2026 11:01:53 -0400 Subject: [PATCH 10/12] Add bugs --- proposals/testing/0025-tag-based-test-execution-filtering.md | 1 + 1 file changed, 1 insertion(+) diff --git a/proposals/testing/0025-tag-based-test-execution-filtering.md b/proposals/testing/0025-tag-based-test-execution-filtering.md index 30d5f3dfaf..d106e90da8 100644 --- a/proposals/testing/0025-tag-based-test-execution-filtering.md +++ b/proposals/testing/0025-tag-based-test-execution-filtering.md @@ -4,6 +4,7 @@ - Authors: [Gustavo Medori](https://github.com/gmedori) * Review Manager: [Paul LeMarquand](https://github.com/plemarquand) * Status: **Active Review (May 21 - June 4, 2026)** +- Bugs: [swiftlang/swift-testing#591](https://github.com/swiftlang/swift-testing/issues/591), rdar://132989780 - Implementation: [swiftlang/swift-testing#1531](https://github.com/swiftlang/swift-testing/pull/1531) - Review: [pitch](https://forums.swift.org/t/pitch-tag-based-test-execution-filtering/86001) From 948cc8138ba0359d15e29987cc3515330c029618 Mon Sep 17 00:00:00 2001 From: Gustavo Medori Date: Thu, 11 Jun 2026 10:13:25 -0600 Subject: [PATCH 11/12] Fix minor typos --- proposals/testing/0025-tag-based-test-execution-filtering.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/testing/0025-tag-based-test-execution-filtering.md b/proposals/testing/0025-tag-based-test-execution-filtering.md index d106e90da8..9d9a137762 100644 --- a/proposals/testing/0025-tag-based-test-execution-filtering.md +++ b/proposals/testing/0025-tag-based-test-execution-filtering.md @@ -67,7 +67,7 @@ As of [SE-0451](https://github.com/swiftlang/swift-evolution/blob/main/proposals @Test func `tag:uiTest`() { /* ... */ } ``` -To continue to allow matching for such function names, I propose introducing a _separate_ prefix called `id:` which behaves much like the `tag:` prefix in that everything that follows it is a regular expression. It's job, however, is to disambiguate and allow the user a mechanism to explicitly say "match on test symbol names please." +To continue to allow matching for such function names, I propose introducing a _separate_ prefix called `id:` which behaves much like the `tag:` prefix in that everything that follows it is a regular expression. Its job, however, is to disambiguate and allow the user a mechanism to explicitly say "match on test symbol names please." ```sh swift test --skip 'id:tag:uiTest' @@ -118,7 +118,7 @@ This introduces a new mechanism that can be used by any existing tools to filter ## Future Directions -Filtering based on tags is quite broad and general purpose. Because you can define any tag to stick on any test or suite, and because tags exists orthogonally to the test graph, you can arbitrarily include/skip any test based solely on the semantics of your tags. However, this change does raise the question of what _else_ we could filter/skip on and how we can be more expressive about it. +Filtering based on tags is quite broad and general purpose. Because you can define any tag to stick on any test or suite, and because tags exist orthogonally to the test graph, you can arbitrarily include/skip any test based solely on the semantics of your tags. However, this change does raise the question of what _else_ we could filter/skip on and how we can be more expressive about it. For example, you may wish to filter/skip tests based on protocol conformance and/or inheritance. A suite's ancestor types can be a useful, and perhaps more natural, signal indicating whether it should run in a given context or not because the ancestor types carry with them behaviors and contracts that have powerful semantic meaning. In the future, we may seek to expand the prefix operators we allow beyond just `tag:`. From ff084d54ac6635da17236028097c17b886462a85 Mon Sep 17 00:00:00 2001 From: Paul LeMarquand Date: Thu, 11 Jun 2026 14:52:05 -0400 Subject: [PATCH 12/12] Update review dates --- proposals/testing/0025-tag-based-test-execution-filtering.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/testing/0025-tag-based-test-execution-filtering.md b/proposals/testing/0025-tag-based-test-execution-filtering.md index 9d9a137762..84c7989a22 100644 --- a/proposals/testing/0025-tag-based-test-execution-filtering.md +++ b/proposals/testing/0025-tag-based-test-execution-filtering.md @@ -3,7 +3,7 @@ - Proposal: [ST-0025](0025-tag-based-test-execution-filtering.md) - Authors: [Gustavo Medori](https://github.com/gmedori) * Review Manager: [Paul LeMarquand](https://github.com/plemarquand) -* Status: **Active Review (May 21 - June 4, 2026)** +* Status: **Active Review (Jun 11 - June 26, 2026)** - Bugs: [swiftlang/swift-testing#591](https://github.com/swiftlang/swift-testing/issues/591), rdar://132989780 - Implementation: [swiftlang/swift-testing#1531](https://github.com/swiftlang/swift-testing/pull/1531) - Review: [pitch](https://forums.swift.org/t/pitch-tag-based-test-execution-filtering/86001)