Skip to content

AI Eyes/Mouth mask components with one-click portrait setup#1330

Closed
SandeepSubba wants to merge 1 commit into
CyberTimon:mainfrom
SandeepSubba:upstream-face-region-masks
Closed

AI Eyes/Mouth mask components with one-click portrait setup#1330
SandeepSubba wants to merge 1 commit into
CyberTimon:mainfrom
SandeepSubba:upstream-face-region-masks

Conversation

@SandeepSubba

Copy link
Copy Markdown

What

Adds Eyes and Mouth AI selections to both the Masks panel and Inpainting, plus a One-Click Portrait button that assembles the classic beauty-retouch stack automatically. The motivating workflow: skin softening needs subject minus eyes minus mouth, which previously meant manual brushing over every eye and lip.

How

DetectionYuNet (MIT, ~230 KB), auto-downloaded and SHA-verified exactly like the existing models. Unlike a plain box detector it returns 5 landmarks per face (eye centers, nose tip, mouth corners), so the masks are anchored on the real features:

  • ellipses centered on the landmarks, sized by interocular / mouth-corner distance, rotated with head tilt
  • input is letterboxed (aspect distortion measurably degrades landmark precision)
  • a second detection pass on the face crop refines the keypoints (the face fills the 640px input instead of a fraction of it)
  • multi-face: every detected face gets its ellipses; per-component Grow/Feather for fine-tuning

Threshold — 0.8, validated against a 64-image RAW sample library: every real face scored >= 0.81 (including tiny background faces, backlit, red stage light, glasses, headwear, tilted heads), while false positives (a bare back, foliage bokeh) topped out at 0.72.

One-Click Portrait (Masks panel) — builds three masks in one press:

Mask UX — visible "+ Add / − Subtract" chips on each mask (subtracting a component was previously three hover-submenu levels deep); Eyes/Mouth are top-level entries in the creation grid and component menus.

Undo correctness — new amendHistory store action folds async generation results (mask bitmaps, generative replace, quick erase) into the creating action's history entry. Previously each AI action left 2–3 history entries, so a single undo left an empty edit behind; now one action — including the whole portrait stack — is exactly one undo step.

Testing

  • Placement verified across all people photos in a 64-image RAW library (frontal, 3/4, tilted, glasses, open-mouth expressions, multiple faces, dark/backlit/colored light) via a pipeline-identical harness, then confirmed in-app
  • cargo check clean; tsc introduces no new errors
  • Single undo after One-Click Portrait removes all three masks; existing edits untouched

🤖 Generated with Claude Code

@SandeepSubba
SandeepSubba requested a review from CyberTimon as a code owner July 5, 2026 00:52
Adds landmark-based Eyes and Mouth selections to masking and inpainting,
plus the retouch workflow built on top of them:

- YuNet face detection (opencv_zoo, MIT, ~230 KB, auto-downloaded like
  the other models): returns face boxes plus 5 landmarks (eye centers,
  nose tip, mouth corners)
- Eyes/Mouth masks are feathered ellipses anchored on the actual
  landmarks, sized by interocular / mouth-corner distance, and rotated
  with head tilt; a second detection pass on the face crop sharpens the
  keypoints considerably; letterboxed input avoids aspect distortion
- Detection threshold 0.8 validated against a 64-image sample library:
  real faces score >= 0.81 (tiny, backlit, red-lit, glasses), false
  positives (bare backs, foliage bokeh) top out at 0.72
- One-Click Portrait: builds Skin Softening (foreground minus eyes and
  mouth), Eye Enhance, and Teeth Whitening (HSL yellows) in one press
- Mask UX: visible "+ Add / - Subtract" chips per mask (subtract was
  previously three hover-submenu levels deep); Eyes/Mouth are top-level
  entries in the creation grid and component menus
- Undo correctness: new amendHistory store action folds async generation
  results into the creating action's history entry, so one user action
  (including the whole portrait stack) is exactly one undo step

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@SandeepSubba
SandeepSubba force-pushed the upstream-face-region-masks branch from 7516b32 to 9f49a62 Compare July 5, 2026 01:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants