Add support for XCUIAutomation

[dfabulich]

https://developer.apple.com/documentation/xcuiautomation

[marcprux]

I do think it would be incredibly useful to have support for an automate-able UI testing (+ screenshot generating) framework.

I've thought about XCUIAutomation, but as I mentioned at #586 (comment), I lean more towards something higher-level like Maestro. I've gotten some rudimentary tests working (both locally and in CI) in a branch of a sample project here: https://github.com/skiptools/skipapp-calculatrix/blob/maestro-tests/Maestro/calculator_test.yaml. There isn't anything Skip-specific in there: given that Skip supports View.accessibilityIdentifier() by passing it through to Compose, stuff like "click the button with id 'foo'" just works.

I'm certainly open to discussing the relative merits of the two approaches, though.

[dfabulich]

Well, here's what I've got. https://github.com/dfabulich/skip-ui-automation

I claim that it would be suitable to move that repo over to the skiptools org.

Step-by-step instructions

Here's what I did:

1. I wrote an XCUIAutomation test in the Showcase app:

   import XCTest
   
   @MainActor
   final class NavigationStackPlaygroundUITests: XCTestCase, Sendable {
       private var app: XCUIApplication!
   
       override func setUp() async throws {
           try await super.setUp()
           continueAfterFailure = false
           app = XCUIApplication()
           app.launchArguments.append("-UITestingResetShowcaseSearch")
           app.launch()
       }
   
       func testNavigationStackPlaygroundShowsPopButton() throws {
           app.tabBars.buttons["Showcase"].tap()
   
           let searchField = app.searchFields.firstMatch
           XCTAssertTrue(searchField.waitForExistence(timeout: 5), "Search field should appear on Showcase tab")
           searchField.tap()
           searchField.typeText("NavigationStack")
   
           let row = app.staticTexts["NavigationStack"]
           XCTAssertTrue(row.waitForExistence(timeout: 5), "Filtered list should include NavigationStack")
           row.tap()
   
           XCTAssertTrue(app.buttons["Pop"].waitForExistence(timeout: 5), "NavigationStack playground should show Pop button")
       }
   }

   I hit a bug with resetting the search field, so I changed @AppStorage("searchText") var searchText = "" to @State var searchText = "" in PlaygroundListView.swift. Then, the test passed.

2. Then, I implemented stubs for the compatibility layer. Cursor helped me import all the @available(*, unavailable) stuff.

3. Then, I transpiled the test by hand. (I put the test in skip-ui/Tests and let the transpiler run; it didn't work at that point because there was no implementation.) I hit a bug transpiling override func setUp() async throws {` so I removed the setup and just put the setup in the test. The end result was:

   package showcase.module
   
   import skip.lib.*
   import skip.unit.*
   import skip.uiautomation.*
   
   @org.junit.runner.RunWith(androidx.test.ext.junit.runners.AndroidJUnit4::class)
   class NavigationStackPlaygroundUITests : XCTestCase {
       @Test
       fun testNavigationStackPlaygroundShowsPopButton() {
           val app = XCUIApplication()
           app.launch()
   
           app.tabBars.buttons["Showcase"].tap()
   
           val searchField = app.searchFields.firstMatch
           XCTAssertTrue(searchField.waitForExistence(timeout = 5.0), "Search field should appear on Showcase tab")
           searchField.tap()
           searchField.typeText("NavigationStack")
   
           val row = app.staticTexts["NavigationStack"]
           XCTAssertTrue(row.waitForExistence(timeout = 5.0), "Filtered list should include NavigationStack")
           row.tap()
   
           XCTAssertTrue(app.buttons["Pop"].waitForExistence(timeout = 5.0), "NavigationStack playground should show Pop button")
       }
   }

   I added it to skipapp-showcase/Android/app/src/androidTest/kotlin/showcase/module/NavigationStackPlaygroundUITests.kt and wired up dependencies. You can see the end result in UI automation skipapp-showcase#80

4. Finally, I wired up an actual implementation based on Android UI Automator (the same technology that Maestro uses under the hood). I needed to add test tags, so I did that in Test tags skip-ui#372 … and then, the test passed.

As for Maestro, Maestro and my thing are both based on Android UI Automator. I think it's inherently useful to have something that can run XCUIAutomation tests as-is, even if we decide that Maestro is better in general.

Speaking for myself, I'm pretty skeptical of Maestro, because Maestro tests are defined in YAML, and I don't love that. My experience with writing "declarative" UI tests is that I'm writing code in a straitjacket. Maestro asks you to write in YAML, switching to JavaScript for "complex logic". But I wanna write Swift.

I think that would be particularly true for my goal, which is to have single-source tests for iOS and Android.