penpot/handoff-ime-bug.md

Handoff — v3 editor IME input (candidate window placement + premature commit)

Context for continuing the IME work on branch poc-japanese-text. Rewritten 2026-07-19 after the premature-commit fix was confirmed with the real IME. Companion to handoff-japanese-text.md (the overall feature handoff); this file covers only the in-flight IME editor work.

Problem statement

When typing Japanese with an IME in the v3 (WASM) text editor:

1. composition must not be interrupted (the user must be able to compose kana and convert to kanji) — FIXED, user-confirmed with real fcitx5-mozc;
2. the IME candidate/suggestion popup must appear next to the current input point, in both horizontal and vertical layouts — partially working; remaining work below.

The user's environment is Linux/X11 + Chromium + fcitx5 with mozc (fcitx5 -d + /usr/lib/mozc/mozc_server; the previous handoff said ibus-mozc — that was wrong).

The two root causes found for premature commit

Real IMEs abort the composition and commit the pending kana on ANY DOM disturbance around the composing element. Two distinct triggers were found and removed:

1. Store dispatch in on-composition-update. sync-wasm-text-editor-content! dispatched dwt/v2-update-text-shape-content per kana → React re-render of the editor component mid-composition (shape content/name/dimensions, foreignObject attrs, CSS vars). Removed: the WASM text_editor_composition_update already writes the preview into WASM shape state and marks it touched, so request-render alone paints the preview. The store is synced once on compositionend.
2. Style writes on the keydown-229 that precedes compositionstart. The old design repositioned the capture surface on the first keydown of an IME sequence (keyCode 229 && !isComposing), believing that was "before the composition". It is not safe: by the time keydown-229 reaches the DOM, the IME has already established its composition context, and the style write still aborts real mozc (CDP tolerates it — see Verification below). The user confirmed the abort persisted after fix 1 alone; it stopped after the redesign in fix 2.

Current design (working tree)

All in frontend/src/app/main/ui/workspace/shapes/text/v3_editor.cljs + v3_editor.scss:

• Idle positioning: the capture surface (#text-editor-wasm-input) sits ON the WASM caret at all times. schedule-ime-caret! runs after every caret-affecting operation (click, pointer-up, double-click, handled keydown, on-input, paste, cut, composition end, focus, mount) — never during a composition (composing-ref guard, set in on-composition-start, cleared in on-composition-end). No handler touches the surface or the store between compositionstart and compositionend.
• schedule-ime-caret! waits two requestAnimationFrames (so the pending WASM render can rebuild the layout the caret rect reads) and then retries up to 30 frames because the rect can be unavailable right after mount. Gotcha: rAF does not fire in occluded/background windows (see Tooling).
• update-ime-caret! (unchanged semantics): horizontal → left/top at caret, font-size = caret rect height; vertical → writing-mode: vertical-rl, right-edge anchor left = (x+width) - origin.x - origin.width, font-size = caret rect width. Returns nil when it could not position (no node/origin/rect) — the scheduler retries on nil.
• pointer-events: none on the surface (v3_editor.scss): clicks always land on the outer overlay div, so offsetX/offsetY remain in the shape's local space regardless of where the surface currently sits. The hover text cursor class moved to the outer div.
• focus-editor!: programmatic .focus() on the surface does NOT reliably fire focus/focusin in Chromium (contenteditable inside SVG foreignObject): the node becomes document.activeElement but React's on-focus never dispatches, so the WASM editor stayed unfocused and every caret-rect read returned null (this made the vertical popup anchor to the default full-size surface with no writing-mode — the "jumping" popup). focus-editor! therefore focuses the node AND calls wasm.api/text-editor-focus + schedule-ime-caret! explicitly. Used by the mount effect and on-pointer-down (which also preventDefaults so mousedown cannot move focus to the body). Note text-editor/text-editor-focus THROWS when the shape id is not in WASM state — a possible mount race to keep in mind.
• on-composition-end: WASM commit + store sync + reset-input-node + schedule-ime-caret! (surface moves to the caret past the committed text). There is no full-size "reset" anymore; the surface stays caret-positioned for the whole editing session.

Supporting changes (unchanged from before, already built):

• frontend/src/app/render_wasm/text_editor.cljs: text-editor-get-cursor-rect reads 4 f32 (x/y/w/h) from _text_editor_get_cursor_rect. Coordinates are ABSOLUTE page coords (selrect origin included); update-ime-caret! subtracts the foreignObject origin (origin-ref, kept fresh each render, includes the valign-adjusted y).
• render-wasm/src/wasm/text_editor.rs: cursor rect independent of the caret blink phase (null only when the editor lacks focus). The WASM artifact with this change is built and loaded in the ws1 devenv.

Verification status

Horizontal — real IME, PASSED (2026-07-19)

Automated real-IME run (see Tooling): typed nihongo, Space-converted, Return-committed through actual fcitx5-mozc:

• ONE composition ｎ→に→にほ→にほんご→日本語, no aborts, no fragment commits; committed text landed in the shape; store synced at compositionend; no crash.
• Surface style byte-identical through the whole composition; a MutationObserver over the entire foreignObject subtree recorded ZERO attribute/childList mutations between compositionstart and compositionend.
• The fcitx candidate window tracked the preedit (window x advanced 866→967 at constant y, matching the DOM caret anchor at the end of the preedit).
• The user separately confirmed typing works with their own keyboard.

Vertical — NOT yet verified with the real IME

The vertical real-IME run was voided: the user's screen was locked (slock) so the synthetic keystrokes never reached the browser (all-blue screenshots, no composition events, text unchanged). Rerun vime-test.sh when the screen is unlocked. Under CDP the vertical mechanics passed earlier (single composition, style byte-identical, commit lands, caret advances down the column), but CDP proves mechanics only.

Remaining known bugs (the actual work left)

1. Horizontal caret rect / hit test broken for CJK-heavy content → popup anchors at the shape origin. With content 日本語ab:
   • text_editor_get_cursor_rect always returns the LINE-START x (rect x == selrect.x) regardless of the real caret offset. In render-wasm/src/wasm/text_editor.rs (get_cursor_rect, ~line 1103): get_rects_for_range(char_pos..char_pos, Tight, Tight) comes back empty and the fallback uses get_glyph_position_at_coordinate((0.0,0.0)) .position as f32 — a glyph INDEX used as an X COORDINATE. That is a bug regardless of the root cause of the empty rects.
   • text_editor_set_cursor_from_offset maps EVERY (x,y) to offset 0 for the same shape (verified with direct WASM calls bypassing all CLJS changes, x swept 1..200, y swept 0..40; valign top). So click-to-place-caret is broken on that content too.
   • Both symptoms suggest the layout the hit-test paths read (text_content.layout.paragraphs) differs from what is painted — possibly laid out without the Japanese fallback fonts (zero-width glyphs collapse every x to 0). The canvas paints correctly, and get_text_dimensions/auto-width grow correctly, so some other layout is fine. Compare how layout.paragraphs is (re)built after edits vs the render path in render/text.rs / render/text_editor.rs.
   • The vertical paths are fine (they recompute from text_vertical::layout_from_content on demand).
2. Vertical popup verification pending (screen lock, above). After the focus fix the mount-time WASM focus is established and the caret rect for vertical returns correct values (measured: end-of-text caret x=3728.6/y=1278/w=16.8/h=1 on the test shape), so the surface should now get writing-mode: vertical-rl + caret position before the first composition. Needs the on-screen run to confirm popup placement.
3. Possible mount race: text-editor-focus throws if the WASM shape is not yet in state when the mount effect runs. Not reproduced conclusively, but one entry attempt showed WASM focus false with no error until a manual call at +300ms succeeded. If editor entry ever silently loses WASM focus again, look here (make focus-editor! retry or make the throw a no-op+retry).
4. Compositions that start WITHOUT keydown (voice input, on-screen keyboards) rely on the last idle position — now always correct by design (the surface is always on the caret), which retires the old concern.
5. v2 editor untouched; macOS/Safari behavior unknown.

Files touched (uncommitted, on poc-japanese-text)

• frontend/src/app/main/ui/workspace/shapes/text/v3_editor.cljs — the design above (update-ime-caret! idle model, schedule-ime-caret! retries, composing-ref, focus-editor!, pointer-events-aware pointer handlers, no store sync in on-composition-update).
• frontend/src/app/main/ui/workspace/shapes/text/v3_editor.scss — pointer-events: none on .text-editor-container.
• frontend/src/app/render_wasm/text_editor.cljs — text-editor-get-cursor-rect.
• render-wasm/src/wasm/text_editor.rs — cursor rect independent of blink phase.

Lint/format for the CLJS file not yet run this round (clj-kondo/cljfmt live in the devenv container, not on the host PATH).

Tooling — REAL-IME automated testing (new, important)

CDP (Input.imeSetComposition) bypasses the OS input method entirely: it cannot reproduce IME aborts and cannot show the candidate popup (an OS window fcitx draws outside the browser — Playwright screenshots never contain it). The real pipeline is scriptable because the Playwright Chrome runs headed on the user's X11 display (DISPLAY=:0):

• Switch engine: fcitx5-remote -s mozc … restore with fcitx5-remote -s keyboard-us-altgr-intl (query with -n).
• Real keystrokes: xdotool windowactivate --sync <WIN> then xdotool type --delay 250 "nihongo", xdotool key space, key Return. XTEST events go through fcitx/mozc exactly like hardware input.
• The Chrome window: find by exact title, e.g. xdotool search --name "Japanese text - Penpot" (was id 60817432; do NOT head -1 a loose pattern).
• Popup observation: full-screen scrot -o file.png + geometry via for w in $(xdotool search --class fcitx); do xdotool getwindowgeometry --shell $w; done. Geometry persists for unmapped windows — corroborate with the screenshot.
• Page-side instrumentation (before typing): composition-event log capturing node.getAttribute('style') and the DOM caret anchor (getSelection().getRangeAt(0).getBoundingClientRect()) per event, plus a MutationObserver over the foreignObject subtree asserting zero attribute/childList mutations mid-composition.
• Scripts saved in the session scratchpad as ime-test.sh (horizontal) and vime-test.sh (vertical) — recreate from this description if gone.

Safety checks before sending synthetic input:

• Screen lock: pgrep slock (user uses slock) and DISPLAY=:0 xset q | grep Monitor (Monitor off = user away). If locked, DO NOT type — keystrokes land in the password prompt.
• The user's fcitx layout must be restored afterwards (keyboard-us-altgr-intl).

Other environment gotchas (all hit this session):

• requestAnimationFrame does not fire while the Chrome window is occluded — schedule-ime-caret! verification MUST be done with the window visible (activate it first). Store-side checks are unaffected.
• After location.reload() the editor may not mount: (reset! app.render-wasm.api/page-transition? false) then re-enter edition.
• Programmatic edition entry: select_shape(uuid) → zoom.fit_to_shapes([uuid]) (NOT the arity-0 fit-to-shapes; Shift+2 via CDP keyboard is unreliable) → start_editing_selected().
• Munged names used a lot: app.render_wasm.text_editor.text_editor_get_cursor_rect, ..._has_focus_QMARK_, app.main.data.workspace.zoom.fit_to_shapes, cljs.core.clj__GT_js. Patching app.render_wasm.text_editor.foo does NOT intercept calls made through app.render_wasm.api.foo (the api def captured the original fn object) — patch the api var too.
• The test shapes contain residue text from aborted/committed test runs: horizontal 0121fd5b-f9b4-8060-8008-586a95941bbe now 日本語ab日本語を知覚異常日本語…, vertical f179b962-b099-8026-8008-5641a910fe2e 日本語をしますかアリアがとうごいます…. Content is junk by design; keep using them.
• File "Japanese text" (Drafts), page showcase, https://localhost:13449/#/workspace?team-id=9aa629b4-0900-8061-8008-3ac96936247d&file-id=f8554703-a493-807a-8008-454b071d651d&page-id=f8554703-a493-807a-8008-454b071d8665.

Suggested next steps (in order)

1. Rerun the vertical real-IME test with the screen unlocked (vime-test.sh); verify the popup follows the caret down the column and commits cleanly. Also re-verify horizontal once more after any changes.
2. Fix the Rust horizontal caret-rect/hit-test for CJK content (bug 1): find why layout.paragraphs yields empty caret-range rects / zero-width positions for 日本語ab while painting is correct; fix the index-as-coordinate fallback while there. Add a Rust regression test with CJK content asserting get_cursor_rect x monotonically increases with the caret offset and set_cursor_from_offset round-trips.
3. Re-verify click-to-place-caret + surface positioning end-to-end in both flows, then run clj-kondo/cljfmt (devenv) on the edited CLJS.
4. Human confirmation from the user in both flows (type, convert, Esc-cancel, two consecutive compositions, click into text mid-composition-free).

See auto-memory v3-editor-repl-verification for the distilled recipes and handoff-japanese-text.md for build/test commands and branch state.