Skip to content

Image-Based Drag Gesture Support #634

Closed

Description

@CCYellowStar2

Describe the feature

Currently, the app allows image-based targeting for click actions but restricts drag gestures to fixed coordinates. This feature proposes extending image recognition capabilities to drag operations, enabling users to:

  1. Define drag start positions using detected images
  2. Define drag end positions using detected images
  3. Optional coordinate fallback if image matching fails

This should behave similarly to existing click-image targeting, but for swipe/drag actions. The controls should appear in the drag gesture configuration section, with visual previews of detected image matches during setup.

Describe the usage

Concrete Example 1: File Management
In Solid Explorer (https://play.google.com/store/apps/details?id=pl.solidexplorer2), automate moving files between folders by:

Detecting the source folder icon as drag start
Detecting the destination folder icon as drag end
Concrete Example 2: Social Media
Automate Instagram Stories browsing (https://instagram.com):

Detect the right-side edge "hot zone" as drag start
Detect left-side edge as drag end for back-swipe

Is it a duplicate?

  • There is no duplicates

Activity

  1. Repository owner locked and limited conversation to collaborators on Jun 22, 2026
  2. converted this issue into a discussion #895 on Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions