diff --git a/.serena/memories/frontend/core.md b/.serena/memories/frontend/core.md index 9afe4e20a7..23ba345f9b 100644 --- a/.serena/memories/frontend/core.md +++ b/.serena/memories/frontend/core.md @@ -37,6 +37,7 @@ See `mem:scripts/paren-repair`. UI and packages: - App UI components, SCSS modules, style-system boundaries, accessibility, i18n, and render performance: `mem:frontend/ui-conventions-and-style-system`. - JS/TS packages, shared UI package, text editor, Storybook, and package builds: `mem:frontend/ui-packages-text-editor-workflow`. +- IME composition in the WASM text editor (capture surface rules, candidate window anchoring, Linux Chrome caret-bounds limit, real-IME debugging): `mem:frontend/text-editor-ime`. - PO translation workflow and per-locale conventions: `mem:frontend/translations`. Workspace behavior: diff --git a/.serena/memories/frontend/text-editor-ime.md b/.serena/memories/frontend/text-editor-ime.md new file mode 100644 index 0000000000..816c4ef39b --- /dev/null +++ b/.serena/memories/frontend/text-editor-ime.md @@ -0,0 +1,28 @@ +# WASM Text Editor IME Input + +IME (Japanese/CJK composition) in the v3 WASM text editor, `app.main.ui.workspace.shapes.text.v3-editor`. The WASM editor (`app.render-wasm.text-editor`, main-thread `_text_editor_*` exports) owns text, caret, and the composition preview; the DOM only captures input. + +## Capture surface model + +- Input lands in a hidden contenteditable (`#text-editor-wasm-input`, opacity 0, `pointer-events:none`) inside the editor `foreignObject`, wrapped by a positioned `.ime-anchor` div. Native IMEs anchor their candidate window to the DOM caret inside it. +- Idle: the surface is placed ON the WASM caret (`update-ime-caret!`) after every caret-affecting operation, collapsed along the inline axis (`scaleX`/`scaleY` ≈ 0.001) so retained text never moves the DOM caret. Vertical text uses `writing-mode:vertical-rl` and sits one column right of the caret column (macOS Chrome/Firefox: on the column). +- Mid-composition: NEVER write the composing element (style, text, selection) and never dispatch to the store; real IMEs (mozc on ibus/fcitx) abort and commit on any such mutation, including style writes on the keydown-229 before `compositionstart`. Only the wrapper moves (`follow-ime-caret!`: transform, or left/top on macOS vertical), by the WASM caret delta from the idle anchor. +- The WASM preview lives in WASM state (`text-editor-composition-update`); the store syncs on `compositionend` only. Preview fallback fonts must be requested explicitly (`load-composition-fonts!`), or uncommitted glyphs render as squares. +- Composition caret placement: on Linux+Chromium (`follow-ime-cursor?`) it tracks the IME's own cursor (the DOM selection, read on `input` with `insertCompositionText`); elsewhere it goes to the end of the changed text (`changed-span-end`) and later cursor-only moves mirror the DOM selection. +- Chromium: programmatic `.focus()` on a contenteditable inside SVG `foreignObject` does not fire focus/focusin; establish WASM focus explicitly (`focus-editor!`). +- If the anchored caret falls outside the `foreignObject` box, Chrome scrolls it (`scrollLeft`) and cancels the offset; vertical text widens the foreignObject, not the clip rect. + +## Platform IME behavior + +- Chrome on Linux X11 (GTK IM path, `ui/gtk/input_method_context_impl_gtk.cc`) passes caret bounds to the IME ONLY inside `DispatchKeyEvent` (key press and release); `SetCursorLocation` just stores them. An anchor move caused by the IME's reply that arrives after the key release (mozc TAB prediction is slow) leaves the candidate window stale until the next key. No page-side workaround: moving the anchor by transform or by layout makes no difference, and synthetic key events never reach the browser process. Keep the anchor still between keys instead. +- Blink side is not the bottleneck: the renderer recomputes selection bounds every main frame (`WidgetBase::WillBeginMainFrame` → `UpdateSelectionBounds`), and a layout-tree update invalidates its cache. +- mozc cursor: end of the preedit while typing; start of the active clause during conversion/prediction (stays there while cycling candidates). The first TAB/Space of a conversion moves it back to the clause start, so that one key lags on Linux Chrome, exactly as in a plain Chrome textarea. +- Synthetic input (`cljs_repl`, CDP `Input.imeSetComposition`) bypasses the OS IME: it validates event flow and WASM preview/commit, but cannot reproduce aborts, candidate-window position, or long-lived uncommitted preedits (real users often leave the preedit uncommitted for seconds). + +## Debugging + +- Geometry overlay: `debug.toggle_debug('ime')` (no reload). Blue = WASM caret, yellow = DOM caret, pink = DOM composition range (what Chromium reports to the IME); values are viewport CSS px. The native candidate window position is not visible to the page. +- When only the user reproduces: install capture-phase document listeners (keydown/keyup/composition*/input) logging `Date.now()`, `getSelection().focusOffset`, the wrapper style and the DOM caret rect; with the page `screenX/screenY`, `devicePixelRatio`, `outerHeight - innerHeight*dpr` (browser chrome height) to map viewport → screen px. Page listeners and `set!` patches vanish on reload; reinstall. +- Real candidate window position on X11/fcitx5: find `Fcitx5 Input Window` with `xdotool search --class fcitx`, then poll `xwininfo -id ` (absolute X/Y, size, Map State) every ~30 ms in a background loop, logging only changes with timestamps; correlate with the page log. Pressing a lone modifier (Ctrl) that snaps a stale window to the right place confirms a key-event-only bounds update. +- Screen recordings (e.g. Peek) of a user repro: extract timestamped contact sheets with `ffmpeg -vf "fps=2,scale=606:-1,drawtext=text='%{pts\:hms}':x=5:y=5:fontsize=20:fontcolor=yellow:box=1:boxcolor=black,tile=2x3"` and read them with the overlay panel visible. +- Driving the real IME from scripts on X11: `fcitx5-remote -s mozc` / restore the previous IM (`-n` to query), `xdotool windowactivate --sync`, `xdotool type --delay 250`, `xdotool key Tab|space|Return`. Check the screen is unlocked and on first, and get the user's consent: it takes over their keyboard. diff --git a/frontend/packages/draft-js/index.js b/frontend/packages/draft-js/index.js index 5372d7ee40..97c4518f70 100644 --- a/frontend/packages/draft-js/index.js +++ b/frontend/packages/draft-js/index.js @@ -34,6 +34,10 @@ function mergeBlockData(block, newData) { let data = block.getData(); for (let key of Object.keys(newData)) { + if (newData[key] == null) { + data = data.delete(key); + continue; + } const oldVal = data.get(key); if (oldVal === newData[key]) { data = data.delete(key); diff --git a/frontend/src/app/main/data/workspace/modifiers.cljs b/frontend/src/app/main/data/workspace/modifiers.cljs index 0e4bb0a5ce..6f440b0a2e 100644 --- a/frontend/src/app/main/data/workspace/modifiers.cljs +++ b/frontend/src/app/main/data/workspace/modifiers.cljs @@ -35,6 +35,7 @@ [app.render-wasm.api :as wasm.api] [app.render-wasm.gesture :as wasm-gesture] [app.render-wasm.shape :as wasm.shape] + [app.util.text.writing-mode :as wm] [beicon.v2.core :as rx] [potok.v2.core :as ptk])) @@ -1129,21 +1130,32 @@ (rx/of (dwu/commit-undo-transaction undo-id)) (rx/empty)))))))) +(defn vertical-text-shape? + [shape] + (wm/vertical-text-content? (get shape :content))) + ;; Pure function to determine next grow-type for text layers (defn next-grow-type - [current-grow-type scalev] - (cond - (= current-grow-type :fixed) - :fixed + ([current-grow-type scalev] + (next-grow-type current-grow-type scalev false)) - (and (not (mth/close? (:y scalev) 1.0)) - (or (= current-grow-type :auto-width) - (= current-grow-type :auto-height))) - :fixed + ([current-grow-type scalev vertical?] + ;; Reuse the horizontal state machine with swapped vertical axes. + (let [scalev (if vertical? + (gpt/point (:y scalev) (:x scalev)) + scalev)] + (cond + (= current-grow-type :fixed) + :fixed - (and (not (mth/close? (:x scalev) 1.0)) - (= current-grow-type :auto-width)) - :auto-height + (and (not (mth/close? (:y scalev) 1.0)) + (or (= current-grow-type :auto-width) + (= current-grow-type :auto-height))) + :fixed - :else - current-grow-type)) + (and (not (mth/close? (:x scalev) 1.0)) + (= current-grow-type :auto-width)) + :auto-height + + :else + current-grow-type)))) diff --git a/frontend/src/app/main/data/workspace/texts.cljs b/frontend/src/app/main/data/workspace/texts.cljs index bc077f92c4..16e52c4c71 100644 --- a/frontend/src/app/main/data/workspace/texts.cljs +++ b/frontend/src/app/main/data/workspace/texts.cljs @@ -24,6 +24,7 @@ [app.common.types.shape.layout :as ctl] [app.common.types.shape.text :as cts] [app.common.types.text :as txt] + [app.common.types.text.cjk-layout :as cjk] [app.common.uuid :as uuid] [app.main.data.changes :as dch] [app.main.data.helpers :as dsh] @@ -441,6 +442,12 @@ [{:keys [attrs shape]}] (shape-current-values shape txt/is-root-node? attrs)) +(defn current-ruby-values + [{:keys [attrs shape]}] + (shape-current-values shape + #(and (txt/is-text-node? %) (cjk/ruby-span? %)) + attrs)) + (defn v3-current-text-values [{:keys [editor-styles attrs]}] (let [result (-> editor-styles @@ -467,7 +474,8 @@ (defn current-paragraph-values [{:keys [editor-styles editor-state editor-instance attrs shape] :as options}] (cond - (some? editor-styles) (v3-current-text-values options) + (some? editor-styles) (merge (shape-current-values shape txt/is-paragraph-node? attrs) + (select-keys editor-styles attrs)) (some? editor-instance) (v2-current-text-values options) (some? editor-state) (v1-current-paragraph-values options) :else (shape-current-values shape txt/is-paragraph-node? attrs))) @@ -505,26 +513,24 @@ (count (:text node))))) - (defn decorate-range-info - "Adds information about ranges inside the metadata of the text nodes" + "Adds information about ranges inside the metadata of the text nodes. + Offsets run top-down so every node counts from its parent's start." [content] - (->> (with-meta content {:start 0 :end (count-node-chars content)}) - (txt/transform-nodes - (fn [node] - (d/update-when - node - :children - (fn [children] - (let [start (-> node meta (:start 0))] - (->> children - (reduce (fn [[result start] node] - (let [end (+ start (count-node-chars node))] - [(-> result - (conj (with-meta node {:start start :end end}))) - end])) - [[] start]) - (first))))))))) + (letfn [(decorate-children [node] + (let [start (-> node meta (:start 0))] + (d/update-when + node + :children + (fn [children] + (->> children + (reduce (fn [[result start] child] + (let [end (+ start (count-node-chars child)) + child (with-meta child {:start start :end end})] + [(conj result (decorate-children child)) end])) + [[] start]) + (first))))))] + (decorate-children (with-meta content {:start 0 :end (count-node-chars content)})))) (defn split-content-at [content position] @@ -559,10 +565,41 @@ (<= (-> node meta :end) end)))) #(d/patch-object % attrs)))) -(defn- update-text-range-attrs +(defn- overlapping-text-nodes + "Text nodes of a range-decorated content with characters in [start, end)." + [content start end] + (->> (txt/node-seq txt/is-text-node? content) + (filter (fn [node] + (let [{node-start :start node-end :end} (meta node)] + (and (< node-start end) (< start node-end))))))) + +(defn- annotated-span-bounds + "The range [start, end) widened to cover every ruby or warichu span it + cuts, so a reading or note is never split between two spans." + [content start end] + (->> (overlapping-text-nodes content start end) + (filter cjk/annotated-span?) + (reduce (fn [[start end] node] + (let [{node-start :start node-end :end} (meta node)] + [(min start node-start) (max end node-end)])) + [start end]))) + +(defn single-span-range? + "True when the characters [start, end) of `content`, widened to whole ruby + and warichu spans, all belong to one text span." + [content start end] + (or (nil? content) + (let [content (decorate-range-info content) + [start end] (annotated-span-bounds content start end)] + (<= (count (overlapping-text-nodes content start end)) 1)))) + +(defn update-text-range-attrs + "Applies `attrs` to the characters [start, end) of a text shape. A range + that cuts a ruby or warichu span covers the whole span." [shape start end attrs] - (let [new-content (-> (:content shape) - (decorate-range-info) + (let [content (decorate-range-info (:content shape)) + [start end] (annotated-span-bounds content start end) + new-content (-> content (split-content-at start) (split-content-at end) (update-content-range start end attrs))] @@ -628,30 +665,32 @@ (defn update-paragraph-attrs [{:keys [id attrs]}] - (let [attrs (d/without-nils attrs)] - (ptk/reify ::update-paragraph-attrs - ptk/UpdateEvent - (update [_ state] - (d/update-in-when state [:workspace-editor-state id] ted/update-editor-current-block-data attrs)) + (ptk/reify ::update-paragraph-attrs + ptk/UpdateEvent + (update [_ state] + (d/update-in-when state [:workspace-editor-state id] ted/update-editor-current-block-data attrs)) - ptk/WatchEvent - (watch [_ state _] - (when-not (some? (get-in state [:workspace-editor-state id])) - (let [objects (dsh/lookup-page-objects state) - shape (get objects id) + ptk/WatchEvent + (watch [_ state _] + (when-not (some? (get-in state [:workspace-editor-state id])) + (let [objects (dsh/lookup-page-objects state) + shape (get objects id) - merge-fn (fn [node attrs] - (reduce-kv - (fn [node k v] (assoc node k v)) - node - attrs)) + merge-fn (fn [node attrs] + (reduce-kv + (fn [node k v] + (if (nil? v) + (dissoc node k) + (assoc node k v))) + node + attrs)) - update-fn #(txt/update-text-content % txt/is-paragraph-node? merge-fn attrs) - shape-ids (cond - (cfh/text-shape? shape) [id] - (cfh/group-shape? shape) (cfh/get-children-ids objects id))] + update-fn #(txt/update-text-content % txt/is-paragraph-node? merge-fn attrs) + shape-ids (cond + (cfh/text-shape? shape) [id] + (cfh/group-shape? shape) (cfh/get-children-ids objects id))] - (rx/of (dwsh/update-shapes shape-ids update-fn)))))))) + (rx/of (dwsh/update-shapes shape-ids update-fn))))))) (defn update-text-attrs [{:keys [id attrs]}] @@ -683,6 +722,47 @@ updated-shape))] (rx/of (dwsh/update-shapes shape-ids merge-shape))))))) +(defn update-ruby-presentation-attrs + [shape attrs] + (txt/update-text-content + shape + #(and (txt/is-text-node? %) (cjk/ruby-span? %)) + d/txt-merge + attrs)) + +(defn update-all-ruby-presentation + "Apply ruby presentation `attrs` to the ruby spans of the text shapes in + `ids`, and of the texts inside the groups among them, in one update." + [ids attrs] + (ptk/reify ::update-all-ruby-presentation + ptk/WatchEvent + (watch [_ state _] + (let [objects (dsh/lookup-page-objects state) + wasm? (features/active-feature? state "render-wasm/v1") + shape-ids (into [] + (comp (mapcat (fn [id] + (let [shape (get objects id)] + (cond + (cfh/text-shape? shape) [id] + (cfh/group-shape? shape) (cfh/get-children-ids objects id))))) + (distinct)) + ids) + update-fn (fn [shape] + (let [updated-shape (update-ruby-presentation-attrs shape attrs)] + (when (and wasm? (cfh/text-shape? updated-shape)) + (wasm.text-editor/cache-shape-text-content! + (:id updated-shape) + (:content updated-shape))) + updated-shape)) + undo-id (js/Symbol)] + (rx/concat + (rx/of (dwu/start-undo-transaction undo-id) + (dwsh/update-shapes shape-ids update-fn)) + (if wasm? + (rx/of (dwwt/resize-wasm-text-all shape-ids)) + (rx/empty)) + (rx/of (dwu/commit-undo-transaction undo-id))))))) + (defn migrate-node [node] (let [color-attrs (not-empty (select-keys node types.fills/fill-attrs))] diff --git a/frontend/src/app/main/data/workspace/transforms.cljs b/frontend/src/app/main/data/workspace/transforms.cljs index 48b74d7023..8e9f457f38 100644 --- a/frontend/src/app/main/data/workspace/transforms.cljs +++ b/frontend/src/app/main/data/workspace/transforms.cljs @@ -256,7 +256,9 @@ ;; Calculate new grow-type for text layers new-grow-type (when (cfh/text-shape? shape) - (dwm/next-grow-type (dm/get-prop shape :grow-type) scalev)) + (dwm/next-grow-type (dm/get-prop shape :grow-type) + scalev + (dwm/vertical-text-shape? shape))) ;; When the horizontal/vertical scale a flex children with auto/fill ;; we change it too fixed @@ -540,7 +542,9 @@ new-height (if (= attr :height) value sr-height) scalev (gpt/point (/ new-width sr-width) (/ new-height sr-height)) current-grow-type (dm/get-prop shape :grow-type) - new-grow-type (dwm/next-grow-type current-grow-type scalev)] + new-grow-type (dwm/next-grow-type current-grow-type + scalev + (dwm/vertical-text-shape? shape))] (cond-> modifiers (not= new-grow-type current-grow-type) (ctm/change-property :grow-type new-grow-type))) diff --git a/frontend/src/app/main/data/workspace/wasm_text.cljs b/frontend/src/app/main/data/workspace/wasm_text.cljs index 9ff79af325..3471e9fc6e 100644 --- a/frontend/src/app/main/data/workspace/wasm_text.cljs +++ b/frontend/src/app/main/data/workspace/wasm_text.cljs @@ -16,6 +16,7 @@ [app.common.geom.point :as gpt] [app.common.render-wasm.api.select :as wselect] [app.common.types.modifiers :as ctm] + [app.common.types.text.cjk-layout :as cjk] [app.main.data.helpers :as dsh] [app.main.data.workspace :as-alias dw] [app.main.data.workspace.modifiers :as dwm] @@ -29,6 +30,31 @@ (def debounce-resize-text-time 40) +(defn resolve-text-size + "Selects the axes controlled by WASM for a text grow type. Vertical + auto-height keeps the physical height as its wrap budget and grows width." + [selrect grow-type content dimension] + (when (or (= :fixed grow-type) (some? dimension)) + (let [vertical? (cjk/vertical-text-content? content)] + {:width (cond + (= :fixed grow-type) + (:width selrect) + + (and (= :auto-height grow-type) (not vertical?)) + (:width selrect) + + :else + (:width dimension)) + :height (cond + (= :fixed grow-type) + (:height selrect) + + (and (= :auto-height grow-type) vertical?) + (:height selrect) + + :else + (:height dimension))}))) + (defn get-wasm-text-new-size "Computes the new {width, height} for a text shape from WASM text layout. For :fixed grow-type, updates WASM content and returns current dimensions (no resize)." @@ -51,14 +77,7 @@ (wasm.api/set-shape-text-images id content)) (let [dimension (when (not= :fixed grow-type) (wasm.api/get-text-dimensions))] - ;; nil dimension = shape not present in WASM state; skip the resize. - (when (or (= :fixed grow-type) (some? dimension)) - {:width (if (#{:fixed :auto-height} grow-type) - (:width selrect) - (:width dimension)) - :height (if (= :fixed grow-type) - (:height selrect) - (:height dimension))}))))) + (resolve-text-size selrect grow-type content dimension))))) (defn resize-wasm-text-modifiers ([shape] diff --git a/frontend/src/app/main/ui/css_cursors.cljs b/frontend/src/app/main/ui/css_cursors.cljs index 8541d7bedf..91f30c1941 100644 --- a/frontend/src/app/main/ui/css_cursors.cljs +++ b/frontend/src/app/main/ui/css_cursors.cljs @@ -17,6 +17,11 @@ [name rotation] (dm/str "cursor-" name "-" (mod (* (.floor js/Math (/ rotation angle-step)) angle-step) 360))) +(defn get-text + "Returns the text cursor class, rotating the I-beam with vertical text flow." + [rotation vertical?] + (get-dynamic "text" (+ (or rotation 0) (if vertical? 90 0)))) + (defn init-static-cursor-style "Initializes a static cursor style" [style name value] diff --git a/frontend/src/app/main/ui/workspace/shapes/text/editor.cljs b/frontend/src/app/main/ui/workspace/shapes/text/editor.cljs index 770a1ebec7..beec0b4883 100644 --- a/frontend/src/app/main/ui/workspace/shapes/text/editor.cljs +++ b/frontend/src/app/main/ui/workspace/shapes/text/editor.cljs @@ -25,6 +25,7 @@ [app.util.keyboard :as kbd] [app.util.object :as obj] [app.util.text-editor :as ted] + [app.util.text.writing-mode :as wm] [goog.events :as events] [rumext.v2 :as mf])) @@ -36,7 +37,7 @@ (let [bprops (obj/get props "blockProps") data (obj/get bprops "data") style (sts/generate-paragraph-styles (obj/get bprops "shape") - (obj/get bprops "data")) + (wm/renderable-paragraph data)) dir (:text-direction data "auto")] @@ -247,7 +248,7 @@ :opacity (when @blurred 0)} :on-pointer-down on-pointer-down :class (dom/classnames - (cur/get-dynamic "text" (:rotation shape)) true + (cur/get-text (:rotation shape) (wm/vertical-text-content? content)) true :align-top (= (:vertical-align content "top") "top") :align-center (= (:vertical-align content) "center") :align-bottom (= (:vertical-align content) "bottom"))} diff --git a/frontend/src/app/main/ui/workspace/shapes/text/ime_debug.cljs b/frontend/src/app/main/ui/workspace/shapes/text/ime_debug.cljs new file mode 100644 index 0000000000..d66c8927d5 --- /dev/null +++ b/frontend/src/app/main/ui/workspace/shapes/text/ime_debug.cljs @@ -0,0 +1,155 @@ +;; This Source Code Form is subject to the terms of the Mozilla Public +;; License, v. 2.0. If a copy of the MPL was not distributed with this +;; file, You can obtain one at http://mozilla.org/MPL/2.0/. +;; +;; Copyright (c) KALEIDOS SUBSIDIARY SL + +(ns app.main.ui.workspace.shapes.text.ime-debug + "Read-only IME geometry overlay. Enable with debug.toggle_debug('ime')." + (:require-macros [app.main.style :as stl]) + (:require + [app.config :as cf] + [app.render-wasm.text-editor :as text-editor] + [app.util.debug :as dbg] + [rumext.v2 :as mf])) + +(defn- screen-rect + [{:keys [x y width height]} ^js matrix] + (let [points (for [[px py] [[x y] [(+ x width) y] + [x (+ y height)] [(+ x width) (+ y height)]]] + [(+ (* (.-a matrix) px) (* (.-c matrix) py) (.-e matrix)) + (+ (* (.-b matrix) px) (* (.-d matrix) py) (.-f matrix))]) + xs (map first points) + ys (map second points) + left (apply min xs) + top (apply min ys)] + {:x left :y top :width (- (apply max xs) left) :height (- (apply max ys) top)})) + +(defn- dom-rect + [^js rect] + {:x (.-x rect) :y (.-y rect) :width (.-width rect) :height (.-height rect)}) + +(defn- text-position + "Resolve a UTF-16 offset, keeping range edges inside the composing text." + [nodes offset edge] + (loop [[node & remaining] nodes + offset offset] + (when node + (if (or (< offset (.-length ^js node)) + (and (= offset (.-length ^js node)) + (or (= edge :end) (empty? remaining)))) + [node offset] + (recur remaining (- offset (.-length ^js node))))))) + +(defn- composition-range + [^js node {:keys [start text]}] + (let [walker (.createTreeWalker js/document node js/NodeFilter.SHOW_TEXT) + nodes (loop [nodes []] + (if-let [text-node (.nextNode walker)] + (recur (conj nodes text-node)) + nodes)) + from (text-position nodes start :start) + to (text-position nodes (+ start (count text)) :end)] + (when (and from to) + (let [range (.createRange js/document)] + (.setStart range (first from) (second from)) + (.setEnd range (first to) (second to)) + range)))) + +(defn- browser-caret + [^js node] + (let [selection (.getSelection js/window)] + (when (and selection (.contains node (.-focusNode selection))) + (let [range (.createRange js/document)] + (.setStart range (.-focusNode selection) (.-focusOffset selection)) + (.collapse range true) + (dom-rect (.getBoundingClientRect range)))))) + +(defn- snapshot + "All bounds use viewport CSS pixels, matching the fixed debug overlay." + [^js node {:keys [active text] :as composition}] + (let [matrix (some-> (.closest node "g.text-editor") (.getScreenCTM)) + caret (text-editor/text-editor-get-cursor-rect) + range (when (and active (seq text)) (composition-range node composition))] + {:platform cf/platform + :browser cf/browser + :writing-mode (.-writingMode (.getComputedStyle js/window node)) + :active active + :text text + :start (:start composition) + :scroll {:x (.-scrollLeft node) :y (.-scrollTop node)} + :wasm-caret (when (and matrix caret) (screen-rect caret matrix)) + :dom-caret (browser-caret node) + :composition (when range (mapv dom-rect (array-seq (.getClientRects range))))})) + +(defn- format-rect + [{:keys [x y width height] :as rect}] + (if rect + (str "x=" (.toFixed x 1) " y=" (.toFixed y 1) + " w=" (.toFixed width 2) " h=" (.toFixed height 2)) + "unavailable")) + +(mf/defc bounds* + {::mf/private true} + [{:keys [rect class]}] + (let [{:keys [x y width height]} rect] + [:g {:class class} + ;; Keep zero-size bounds visible without changing the measured values. + [:rect {:x x :y y :width (max 1 width) :height (max 1 height) + :fill "none" :stroke "currentColor" :stroke-width 1}] + [:path {:d (str "M " (- x 4) " " y " h 8 M " x " " (- y 4) " v 8") + :stroke "currentColor" :stroke-width 1}]])) + +(mf/defc overlay* + [{:keys [input-ref composing-ref composition-text-ref composition-base-ref]}] + (let [options (mf/deref dbg/state) + enabled (contains? options :ime) + sample* (mf/use-state nil)] + ;; A separate component owns debug state. Sampling never changes the input, + ;; its selection, or the editor component's state during composition. + (mf/use-effect + (mf/deps enabled) + (fn [] + (when enabled + (let [frame (atom nil)] + (letfn [(tick [] + (when-let [node (mf/ref-val input-ref)] + (let [composition {:active (mf/ref-val composing-ref) + :text (mf/ref-val composition-text-ref) + :start (mf/ref-val composition-base-ref)} + next-sample (snapshot node composition)] + (swap! sample* #(if (= % next-sample) % next-sample)))) + (reset! frame (js/requestAnimationFrame tick)))] + (tick) + (fn [] (js/cancelAnimationFrame @frame))))))) + + (when enabled + (let [{:keys [platform browser writing-mode active text start scroll + wasm-caret dom-caret composition]} @sample*] + ;; A body portal avoids the text shape's SVG clip and transforms. + (mf/portal + (mf/html + [:div {:class (stl/css :ime-debug) :data-testid "ime-debug" :aria-hidden true} + [:svg {:class (stl/css :geometry)} + (when wasm-caret + [:> bounds* {:rect wasm-caret :class (stl/css :wasm-caret)}]) + (when dom-caret + [:> bounds* {:rect dom-caret :class (stl/css :dom-caret)}]) + (for [[index rect] (map-indexed vector composition)] + [:> bounds* {:key index :rect rect :class (stl/css :composition)}])] + [:div {:class (stl/css :panel)} + [:div (str "IME · " (name (or platform cf/platform)) + " · " (name (or browser cf/browser)) " · " writing-mode + " · " (if active "composing" "idle"))] + [:div {:class (stl/css :wasm-caret)} (str "Penpot caret: " (format-rect wasm-caret))] + [:div {:class (stl/css :dom-caret)} (str "Browser caret: " (format-rect dom-caret))] + [:div {:class (stl/css :composition)} + (str "Browser composition: " (format-rect (first composition)) + " · " (count composition) " rectangle(s)")] + [:div (str "Input scroll: " (:x scroll) ", " (:y scroll) + " · composition start: " start " UTF-16 units")] + (when active + [:div {:class (stl/css :composition-text)} (str "Text: " text)]) + [:div {:class (stl/css :note)} + "Bounds are in viewport CSS pixels. Native candidate window bounds are unavailable."]]]) + (.-body js/document)))))) diff --git a/frontend/src/app/main/ui/workspace/shapes/text/ime_debug.scss b/frontend/src/app/main/ui/workspace/shapes/text/ime_debug.scss new file mode 100644 index 0000000000..1212086c6d --- /dev/null +++ b/frontend/src/app/main/ui/workspace/shapes/text/ime_debug.scss @@ -0,0 +1,48 @@ +.ime-debug { + position: fixed; + inset: 0; + z-index: 10000; + pointer-events: none; +} + +.geometry { + width: 100%; + height: 100%; + overflow: visible; +} + +.panel { + position: absolute; + bottom: 48px; + left: 64px; + max-width: 480px; + padding: 8px 12px; + border: 1px solid #697582; + border-radius: 4px; + background: #17212b; + color: #fff; + font: 12px / 18px monospace; +} + +.wasm-caret { + color: #36bdff; +} + +.dom-caret { + color: #ffbe55; +} + +.composition { + color: #ff73ce; +} + +.composition-text { + max-height: 72px; + overflow: hidden; + overflow-wrap: anywhere; + white-space: pre-wrap; +} + +.note { + color: #c0ccd8; +} diff --git a/frontend/src/app/main/ui/workspace/shapes/text/v3_editor.cljs b/frontend/src/app/main/ui/workspace/shapes/text/v3_editor.cljs index 403f6c3030..09698b1901 100644 --- a/frontend/src/app/main/ui/workspace/shapes/text/v3_editor.cljs +++ b/frontend/src/app/main/ui/workspace/shapes/text/v3_editor.cljs @@ -9,6 +9,7 @@ (:require-macros [app.main.style :as stl]) (:require [app.common.data.macros :as dm] + [app.common.types.text.cjk-layout :as cjk] [app.config :as cf] [app.main.data.helpers :as dsh] [app.main.data.workspace :as dw] @@ -17,6 +18,7 @@ [app.main.refs :as refs] [app.main.store :as st] [app.main.ui.css-cursors :as cur] + [app.main.ui.workspace.shapes.text.ime-debug :as ime-debug] [app.render-wasm.api :as wasm.api] [app.render-wasm.text-editor :as text-editor] [app.util.dom :as dom] @@ -174,12 +176,162 @@ Mousetrap's `stopCallback` drops every keystroke whose target is contentEditable, so without the `mousetrap` class (as in V1/V2) the text - shortcuts (Ctrl+B, Ctrl+I, …) never reach the dispatcher." - [rotation] - (dm/str "mousetrap " - (cur/get-dynamic "text" rotation) - " " - (stl/css :text-editor-container))) + shortcuts (Ctrl+B, Ctrl+I, …) never reach the dispatcher. The hover cursor + lives on the overlay: the surface is pointer-events:none." + [] + (dm/str "mousetrap " (stl/css :text-editor-container))) + +(def ^:private ime-surface-collapse + "Inline-axis scale of the capture surface to limit DOM caret drift." + 0.001) + +(defn- macos-vertical-ime-scale + "Keep composition characters nonempty after Chromium rounds screen bounds." + [^js node font-size] + (let [matrix (some-> node (.closest "foreignObject") (.getScreenCTM)) + zoom (if matrix (js/Math.hypot (.-c matrix) (.-d matrix)) 1)] + ;; Chromium derives vertical character height from integer caret positions. + ;; scaleY(0.001) makes most of those heights zero, leaving the native macOS + ;; IME without usable character bounds. Allow 1.5 screen pixels per character, + ;; including a margin for fractional layout/zoom rounding. + (max ime-surface-collapse (/ 1.5 (max 0.001 (* font-size zoom)))))) + +(defn- update-ime-caret! + "Move the hidden capture surface onto the WASM caret so the IME candidate + window opens next to the edited text. Returns the caret rect with its + writing direction, or nil when there is none yet. + + Call only while idle, never during or right before a composition: real + IMEs (ibus-mozc) abort and commit on any mutation of the composing + element, including style writes on the keydown-229 before + compositionstart. So the surface follows the caret after every + caret-affecting operation, and during a composition only its wrapper + moves (see `follow-ime-caret!`). + + The IME anchors its window to the DOM caret inside the surface. The + surface is compressed along the inline axis to limit the advance of its + retained text (see `keep-input-alive`) and composition; the block axis + keeps the caret cell size. macOS vertical characters retain nonempty + screen bounds so its native IME can anchor to them. In vertical text the + surface (`vertical-rl`, `line-height: 1`) aligns its caret cell with the + text column on macOS, except in Safari. Safari has been observed to show + horizontal candidates for vertical text, so it keeps the surface right + of the column, like other platforms whose windows open below the caret. + + `origin` holds the pointer overlay's top-left and width, in the + same page space as the caret rect." + [^js node ^js wrapper origin] + (when (and (some? node) (some? origin)) + (when-let [{:keys [x y width height] :as rect} (text-editor/text-editor-get-cursor-rect)] + (let [style (.-style node) + macos-vertical? (and (:vertical? origin) (cf/check-platform? :macos))] + ;; Unlike overflow:hidden, clip is not a scroll container. Long + ;; compositions must not scroll the capture surface away from its anchor. + (set! (.-overflow style) (if macos-vertical? "clip" "")) + (if (:vertical? origin) + (do + (set! (.-writingMode style) "vertical-rl") + (set! (.-fontSize style) (dm/str width "px")) + (set! (.-lineHeight style) "1") + (set! (.-transformOrigin style) "100% 0") + (set! (.-transform style) + (dm/str "scaleY(" (if macos-vertical? + (macos-vertical-ime-scale node width) + ime-surface-collapse) ")")) + ;; vertical-rl puts the DOM caret in the surface's rightmost cell. + ;; Safari's horizontal candidate window needs an anchor beside + ;; the column. Chrome and Firefox use the column itself on macOS. + (let [right (if (and macos-vertical? (not (cf/check-browser? :safari))) + (+ x width) + (+ x (* 2.15 width)))] + (set! (.-left style) (dm/str (- right (:x origin) (:width origin)) "px")))) + (do + (set! (.-writingMode style) "") + (set! (.-fontSize style) (dm/str height "px")) + (set! (.-lineHeight style) "") + (set! (.-transformOrigin style) "0 0") + (set! (.-transform style) (dm/str "scaleX(" ime-surface-collapse ")")) + (set! (.-left style) (dm/str (- x (:x origin)) "px")))) + (set! (.-top style) (dm/str (- y (:y origin)) "px")) + (when (some? wrapper) + (set! (.. wrapper -style -transform) "") + (set! (.. wrapper -style -left) "") + (set! (.. wrapper -style -top) "")) + (assoc rect :vertical? (:vertical? origin)))))) + +(defn- ime-overlay-padding + "Space around the text overlay for the hidden vertical IME anchor." + [vertical? viewport-width] + (cond + (not vertical?) {:left 0 :right 0} + (cf/check-platform? :macos) {:left viewport-width + :right (if (cf/check-browser? :safari) viewport-width 0)} + :else {:left 0 :right viewport-width})) + +(defn- text-before-selection + "UTF-16 length of the surface text before the DOM selection focus, or nil + when the selection is not inside `node`. Reading it mutates nothing." + [^js node] + (let [sel (.getSelection js/window)] + (when (and (some? node) (some? sel) (pos? (.-rangeCount sel)) + (.contains node (.-focusNode sel))) + (let [range (.createRange js/document)] + (.setStart range node 0) + (.setEnd range (.-focusNode sel) (.-focusOffset sel)) + (.-length (.toString range)))))) + +(defn changed-span-end + "Code point index, in `new-text`, where the part that differs from + `old-text` ends: a newly typed kana or a newly picked candidate." + [old-text new-text] + (let [old (js/Array.from old-text) + new (js/Array.from new-text) + n-old (.-length old) + n-new (.-length new) + prefix (loop [i 0] + (if (and (< i n-old) (< i n-new) (= (aget old i) (aget new i))) + (recur (inc i)) + i)) + suffix (loop [i 0] + (if (and (< i (- n-old prefix)) + (< i (- n-new prefix)) + (= (aget old (- n-old 1 i)) (aget new (- n-new 1 i)))) + (recur (inc i)) + i))] + (- n-new suffix))) + +(def ^:private follow-ime-cursor? + "True where the composition caret tracks the IME cursor (the DOM selection) + instead of the end of the changed text. + + Chromium on Linux hands the caret bounds to GTK input methods only while + dispatching a key event. A caret move caused by the IME's reply arrives + after the key release when the IME is slow (mozc prediction on Tab), so + the candidate window keeps its old position until the next key. The IME + cursor stays on the active clause while cycling candidates, so the window + never needs that late update." + (and (cf/check-platform? :linux) + (or (cf/check-browser? :chrome) + (cf/check-browser? :edge)))) + +(defn- follow-ime-caret! + "Move the surface wrapper by how far the WASM caret moved from `anchor`, + the caret rect the surface was placed on. Writes only the wrapper, never the + composing element, so it is safe mid-composition. Returns nil when the caret + rect is not available yet." + [^js wrapper anchor] + (when (and (some? wrapper) (some? anchor)) + (when-let [{:keys [x y]} (text-editor/text-editor-get-cursor-rect)] + (let [dx (- x (:x anchor)) + dy (- y (:y anchor))] + (if (and (:vertical? anchor) (cf/check-platform? :macos)) + (do + ;; Move the wrapper in layout to update the composition bounds + ;; that macOS uses to position the native candidate window. + (set! (.. wrapper -style -left) (dm/str dx "px")) + (set! (.. wrapper -style -top) (dm/str dy "px"))) + (set! (.. wrapper -style -transform) + (dm/str "translate(" dx "px, " dy "px)"))))))) (mf/defc text-editor* "Contenteditable element positioned over the text shape to capture input events." @@ -188,6 +340,8 @@ clip-id (dm/str "text-edition-clip" shape-id) + vertical? (cjk/vertical-text-content? (:content shape)) + contenteditable-ref (mf/use-ref nil) ;; Number of characters the browser is about to replace via marked text @@ -202,11 +356,30 @@ deferred-press-ref (mf/use-ref nil) + ;; Pointer overlay top-left and width in page space (see `update-ime-caret!`). + origin-ref (mf/use-ref nil) + + ;; True between compositionstart and compositionend; the surface must not move then. + composing-ref (mf/use-ref false) + + ;; Surface wrapper, moved during a composition (see `follow-ime-caret!`). + wrapper-ref (mf/use-ref nil) + + ;; Caret rectangle the surface was last placed on while idle. + anchor-ref (mf/use-ref nil) + + ;; Surface UTF-16 length before the composition, and the composition text. + composition-base-ref (mf/use-ref 0) + composition-text-ref (mf/use-ref "") + + ;; IME cursor offset last read from the DOM selection; nil after a text change. + synced-offset-ref (mf/use-ref nil) + fallback-fonts (wasm.api/fonts-from-text-content (:content shape) false) fallback-families (map (fn [font] (font-family-from-font-id (:font-id font))) fallback-fonts) - [{:keys [x y width height]} transform] + [{:keys [x y width height ime-width ime-left]} transform] (let [{:keys [width height]} (wasm.api/get-text-dimensions shape-id) selrect-transform (mf/deref refs/workspace-selrect) vbox (mf/deref refs/vbox) @@ -224,41 +397,163 @@ overlay-width (if (= (:grow-type shape) :auto-width) (+ max-width viewport-width) max-width) + ime-padding (ime-overlay-padding vertical? viewport-width) valign (-> shape :content :vertical-align) y (:y selrect) y (case valign "bottom" (+ y (- selrect-height height)) "center" (+ y (/ (- selrect-height height) 2)) y)] - [(assoc selrect :y y :width overlay-width :height max-height) transform]) + ;; Widen the foreignObject (not the clip) toward the IME anchor, or + ;; the browser scrolls to reveal the caret and cancels the offset. + [(assoc selrect + :y y :width overlay-width :height max-height + :ime-width (+ overlay-width (:right ime-padding)) + :ime-left (:left ime-padding)) + transform]) + + schedule-ime-caret! + (mf/use-fn + (fn [] + ;; Wait two frames for the pending WASM render to rebuild the layout + ;; the caret rect reads, then retry: after mount the rect can stay + ;; unavailable until the first full layout. Skip while composing. + (letfn [(attempt [tries] + (when-not (mf/ref-val composing-ref) + (if-let [rect (update-ime-caret! + (mf/ref-val contenteditable-ref) + (mf/ref-val wrapper-ref) + (mf/ref-val origin-ref))] + (mf/set-ref-val! anchor-ref rect) + (when (pos? tries) + (js/requestAnimationFrame #(attempt (dec tries)))))))] + (js/requestAnimationFrame + (fn [] + (js/requestAnimationFrame #(attempt 30))))))) + + schedule-ime-follow! + (mf/use-fn + (fn [] + ;; Same wait and retries as `schedule-ime-caret!`: a preview update + ;; clears the layout the caret rect reads until the next render. + ;; Runs only mid-composition. + (letfn [(attempt [tries] + (when (and (mf/ref-val composing-ref) + (nil? (follow-ime-caret! + (mf/ref-val wrapper-ref) + (mf/ref-val anchor-ref))) + (pos? tries)) + (js/requestAnimationFrame #(attempt (dec tries)))))] + (js/requestAnimationFrame + (fn [] + (js/requestAnimationFrame #(attempt 30))))))) + + ;; Moves the WASM caret inside the preview, and the IME window with it. + ;; Synchronous: the browser reports caret bounds to the IME right after + ;; the update that triggered this. + place-composition-caret! + (mf/use-fn + (fn [offset] + (text-editor/text-editor-set-composition-cursor offset) + (when (nil? (follow-ime-caret! (mf/ref-val wrapper-ref) (mf/ref-val anchor-ref))) + (schedule-ime-follow!)))) + + ;; Mirrors the IME cursor (the DOM selection) on the WASM caret when only + ;; the cursor moves, e.g. switching clauses. The first read after a text + ;; change only sets the baseline (see on-composition-update), except + ;; where the caret always tracks the IME cursor (see `follow-ime-cursor?`). + sync-composition-cursor! + (mf/use-fn + (fn [] + (when (mf/ref-val composing-ref) + (when-let [focus (text-before-selection (mf/ref-val contenteditable-ref))] + (let [text (mf/ref-val composition-text-ref) + units (-> (- focus (mf/ref-val composition-base-ref)) + (max 0) + (min (count text))) + cursor (.-length (js/Array.from (subs text 0 units))) + previous (mf/ref-val synced-offset-ref)] + (mf/set-ref-val! synced-offset-ref cursor) + (when (if (some? previous) + (not= cursor previous) + follow-ime-cursor?) + (place-composition-caret! cursor) + (wasm.api/render-text-editor-overlay!))))))) + + ;; In Chromium, .focus() on a contenteditable inside an SVG foreignObject + ;; does not reliably fire focus/focusin, so React's on-focus never runs + ;; and every caret-rect read is null. This sets the WASM focus itself. + focus-editor! + (mf/use-fn + (mf/deps shape-id) + (fn [] + (when-let [node (mf/ref-val contenteditable-ref)] + (when (not= (.-activeElement js/document) node) + (.focus node))) + ;; Re-focusing resets transient WASM editor state (overtype mode, + ;; pending events), so only do it when this shape lacks the focus. + (when-not (and (text-editor/text-editor-has-focus?) + (= shape-id (text-editor/text-editor-get-active-shape-id))) + (wasm.api/text-editor-focus shape-id)) + (schedule-ime-caret!))) on-composition-start (mf/use-fn (fn [_event] + (mf/set-ref-val! composing-ref true) + (mf/set-ref-val! composition-text-ref "") + (mf/set-ref-val! synced-offset-ref nil) + (mf/set-ref-val! composition-base-ref + (let [node (mf/ref-val contenteditable-ref)] + (or (text-before-selection node) + (count (.-textContent node))))) ;; IME composition supplies its own text; drop any pending caret style. (text-editor/clear-pending-caret-styles!) (text-editor/text-editor-composition-start))) on-composition-update (mf/use-fn + (mf/deps shape-id) (fn [event] - ;; IME cancel (e.g. Escape on Linux ibus-mozc) fires compositionupdate - ;; with an empty string; that must reach WASM to clear the preview text. + ;; IME cancel (e.g. Escape in ibus-mozc) fires compositionupdate with + ;; an empty string; WASM needs it to clear the preview. + ;; + ;; The browser owns the capture surface mid-composition, so this + ;; handler must not touch it: no clearing, no style writes, no store + ;; dispatch. A store sync re-renders this component, and that makes + ;; a real IME commit the pending kana and restart (typing "ni" + ;; commits ん then い, not に). WASM keeps the preview in its own + ;; state; the store syncs on compositionend. Only the surface wrapper + ;; moves, so the IME window follows the WASM caret. + ;; + ;; The caret goes to the end of the changed part: a typed kana, a + ;; converted word, or a candidate picked in any clause. Where it + ;; tracks the IME cursor instead, `on-input` places it once the DOM + ;; selection holds that cursor. (let [data (.-data event)] (when (some? data) - (text-editor/text-editor-composition-update data) - (sync-wasm-text-editor-content!) - (wasm.api/request-render-preserving-target "text-composition")) - (reset-input-node (mf/ref-val contenteditable-ref))))) + (let [previous (mf/ref-val composition-text-ref)] + (mf/set-ref-val! composition-text-ref data) + (mf/set-ref-val! synced-offset-ref nil) + (text-editor/text-editor-composition-update data) + ;; The preview never reaches the store, so request its + ;; fallback faces here or it shows missing glyphs until commit. + (wasm.api/load-composition-fonts! shape-id data) + (when-not follow-ime-cursor? + (place-composition-caret! (changed-span-end previous data))) + (wasm.api/request-render-preserving-target "text-composition")))))) on-composition-end (mf/use-fn (fn [^js event] + (mf/set-ref-val! composing-ref false) (let [data (or (.-data event) "")] (text-editor/text-editor-composition-end data) (sync-wasm-text-editor-content!) (wasm.api/request-render-preserving-target "text-composition")) - (reset-input-node (mf/ref-val contenteditable-ref)))) + (reset-input-node (mf/ref-val contenteditable-ref)) + ;; Move the surface onto the caret, now past the committed text. + (schedule-ime-caret!))) on-paste (mf/use-fn @@ -267,7 +562,8 @@ (when-let [data (.-clipboardData event)] (st/emit! (dwt/v3-paste (.getData data "text/html") (.getData data "text/plain")))) - (reset-input-node (mf/ref-val contenteditable-ref)))) + (reset-input-node (mf/ref-val contenteditable-ref)) + (schedule-ime-caret!))) on-copy (mf/use-fn @@ -287,11 +583,16 @@ (text-editor/text-editor-delete-backward) (sync-wasm-text-editor-content!) (wasm.api/request-render-preserving-target "text-cut")) - (reset-input-node (mf/ref-val contenteditable-ref))))) + (reset-input-node (mf/ref-val contenteditable-ref)) + (schedule-ime-caret!)))) on-key-down (mf/use-fn (fn [^js event] + ;; IME keydowns (keyCode 229) must not touch the surface: the IME + ;; already owns it, and even a style write before compositionstart + ;; makes ibus abort and commit. The surface already sits on the + ;; caret (see `update-ime-caret!`). (when (and (text-editor/text-editor-has-focus?) (not (composing-event? event))) (let [key (.-key event) @@ -388,10 +689,12 @@ (text-editor/text-editor-move-cursor 5 ctrl? shift?) (wasm.api/render-text-editor-overlay!)) - ;; Let contenteditable handle text input via on-input, which - ;; needs an editable browser selection. - :else - (ensure-input-caret (mf/ref-val contenteditable-ref))))))) + ;; Let contenteditable handle text input via on-input with an editable selection. + :else (ensure-input-caret (mf/ref-val contenteditable-ref))) + ;; A handled key may move the caret; keep the surface on it for + ;; the next IME sequence. Character keys also reschedule from + ;; on-input after the insert. + (schedule-ime-caret!))))) ;; Native `beforeinput` listener (see the use-effect that registers it). ;; We use the native event, not React's synthetic `onBeforeInput`, because @@ -425,6 +728,9 @@ (let [native-event (.-nativeEvent event) input-type (.-inputType native-event) data (.-data native-event)] + ;; The DOM now holds the updated composition and its IME cursor. + (when (= input-type "insertCompositionText") + (sync-composition-cursor!)) ;; Skip composition-related input events - composition-end handles those (when (and (not (composing-event? event)) (not= input-type "insertCompositionText")) @@ -447,11 +753,17 @@ ;; IMPORTANT: do NOT clear the surface here (see keep-input-alive): ;; the browser must retain the just-typed character so the macOS ;; accent menu can replace it on the next input. - (keep-input-alive (mf/ref-val contenteditable-ref)))))) + (keep-input-alive (mf/ref-val contenteditable-ref)) + (schedule-ime-caret!))))) on-pointer-down (mf/use-fn (fn [^js event] + ;; The capture surface is pointer-events:none, so focus it here. + ;; preventDefault stops mousedown from moving focus to the body, + ;; which would blur the surface. + (dom/prevent-default event) + (focus-editor!) (when-not (secondary-button? event) (let [native-event (dom/event->native-event event) off-pt (dom/get-offset-position native-event)] @@ -496,7 +808,8 @@ ;; Without a drag there is no pointer selection to close; the ;; caret is placed by `on-click`. (when dragging? - (wasm.api/render-text-editor-overlay!)))))) + (wasm.api/render-text-editor-overlay!)) + (schedule-ime-caret!))))) on-click (mf/use-fn @@ -520,7 +833,8 @@ :else (do (wasm.api/text-editor-set-cursor-from-offset off-pt) - (wasm.api/render-text-editor-overlay!))))))) + (wasm.api/render-text-editor-overlay!))) + (schedule-ime-caret!))))) on-double-click (mf/use-fn @@ -529,7 +843,8 @@ off-pt (dom/get-offset-position native-event)] (wasm.api/text-editor-select-word-boundary off-pt) (ensure-input-caret (mf/ref-val contenteditable-ref)) - (wasm.api/render-text-editor-overlay!)))) + (wasm.api/render-text-editor-overlay!) + (schedule-ime-caret!)))) on-context-menu (mf/use-fn @@ -548,7 +863,8 @@ ;; on every other path that moves it. (text-editor/clear-pending-caret-styles!) (wasm.api/text-editor-set-cursor-from-offset off-pt) - (wasm.api/render-text-editor-overlay!))) + (wasm.api/render-text-editor-overlay!) + (schedule-ime-caret!))) ;; Deferred: the dropdown closes itself on a document `contextmenu`, ;; which would close the menu this very event is opening. (ts/schedule @@ -560,7 +876,8 @@ on-focus (mf/use-fn (fn [^js _event] - (wasm.api/text-editor-focus shape-id))) + (wasm.api/text-editor-focus shape-id) + (schedule-ime-caret!))) on-blur (mf/use-fn @@ -573,10 +890,18 @@ (wasm.api/text-editor-blur)))) style #js {:pointerEvents "all" + ;; Keep pointer offsets relative to the text shape when + ;; the foreignObject extends left for the macOS IME anchor. + :position "relative" + :left (dm/str ime-left "px") + :width (dm/str ime-width "px") "--editor-container-width" (dm/str width "px") "--editor-container-height" (dm/str height "px") "--fallback-families" (if (seq fallback-families) (dm/str (str/join ", " fallback-families)) "sourcesanspro")}] + ;; Keep the caret-translation origin in sync with the current render. + (mf/set-ref-val! origin-ref {:x x :y y :width ime-width :vertical? vertical?}) + ;; Exit on Escape via a document key-up listener (like v2). On key-down the trailing ;; key-up is read as a non-editing Escape and deselects the shape. (mf/use-effect @@ -593,6 +918,14 @@ (.addEventListener js/document "keyup" on-key-up) #(.removeEventListener js/document "keyup" on-key-up)))) + ;; The IME cursor can move inside the composition without changing its + ;; text (e.g. switching clauses), which only shows as a selection change. + (mf/use-effect + (mf/deps sync-composition-cursor!) + (fn [] + (.addEventListener js/document "selectionchange" sync-composition-cursor!) + #(.removeEventListener js/document "selectionchange" sync-composition-cursor!))) + ;; Register the native `beforeinput` listener. React's synthetic ;; `onBeforeInput` does not expose `getTargetRanges()`, even with ;; nativeEvent (it's fully synthetic, composed of other two events). @@ -613,9 +946,10 @@ ;; Group the whole editing session (edits, reflow resizes, finalize) into a single ;; undo entry. Nested transactions (e.g. style shortcuts) are ref-counted and fold in. (st/emit! (dwu/start-undo-transaction shape-id :timeout nil)) - (when-let [node (mf/ref-val contenteditable-ref)] - ;; Focus and select all text on mount (this will trigger on-focus) - (.focus node) + (when (some? (mf/ref-val contenteditable-ref)) + ;; Focus and select all text on mount. `focus-editor!` establishes the + ;; WASM focus explicitly; select-all is a no-op without it. + (focus-editor!) (text-editor/text-editor-select-all) (wasm.api/request-render-preserving-target "text-editor-select-all-on-mount")) ;; On unmount, finalize the editor content and then dispose the WASM editor. @@ -654,40 +988,46 @@ [:g.text-editor {:clip-path (dm/fmt "url(#%)" clip-id) :transform (dm/str transform) :data-testid "text-editor"} + [:> ime-debug/overlay* {:input-ref contenteditable-ref + :composing-ref composing-ref + :composition-text-ref composition-text-ref + :composition-base-ref composition-base-ref}] [:defs [:clipPath {:id clip-id} [:rect {:x x :y y :width width :height height}]]] - [:foreignObject {:x x :y y :width width :height height} + [:foreignObject {:x (- x ime-left) :y y :width (+ ime-width ime-left) :height height} [:div {:on-click on-click :on-double-click on-double-click :on-pointer-down on-pointer-down :on-pointer-move on-pointer-move :on-pointer-up on-pointer-up :on-context-menu on-context-menu - :class (stl/css :text-editor) + ;; The hover cursor lives here: the capture surface is pointer-events:none. + :class (dm/str (cur/get-text (:rotation shape) vertical?) + " " + (stl/css :text-editor)) :style style} - [:div - {:ref contenteditable-ref - :contentEditable true - :suppressContentEditableWarning true - ;; The surface retains typed text between keystrokes (see - ;; keep-input-alive), so disable text assistance that would otherwise - ;; rewrite that retained text and desync the WASM editor. - ;; NOTE: this was already not working in v1/v2 - :spellCheck false - :autoCorrect "off" - :autoCapitalize "off" - :on-composition-start on-composition-start - :on-composition-update on-composition-update - :on-composition-end on-composition-end - :on-key-down on-key-down - :on-input on-input - :on-paste on-paste - :on-copy on-copy - :on-cut on-cut - :on-focus on-focus - :on-blur on-blur - :id "text-editor-wasm-input" - :class (input-surface-class (:rotation shape)) - :data-testid "text-editor-container"}]]]])) + [:div {:ref wrapper-ref + :class (stl/css :ime-anchor)} + [:div + {:ref contenteditable-ref + :contentEditable true + :suppressContentEditableWarning true + ;; Text assistance would rewrite the retained text (see keep-input-alive). + :spellCheck false + :autoCorrect "off" + :autoCapitalize "off" + :on-composition-start on-composition-start + :on-composition-update on-composition-update + :on-composition-end on-composition-end + :on-key-down on-key-down + :on-input on-input + :on-paste on-paste + :on-copy on-copy + :on-cut on-cut + :on-focus on-focus + :on-blur on-blur + :id "text-editor-wasm-input" + :class (input-surface-class) + :data-testid "text-editor-container"}]]]]])) diff --git a/frontend/src/app/main/ui/workspace/shapes/text/v3_editor.scss b/frontend/src/app/main/ui/workspace/shapes/text/v3_editor.scss index 5659b0646e..a28d4ddcea 100644 --- a/frontend/src/app/main/ui/workspace/shapes/text/v3_editor.scss +++ b/frontend/src/app/main/ui/workspace/shapes/text/v3_editor.scss @@ -9,4 +9,20 @@ opacity: 0; overflow: hidden; white-space: pre; + + // The surface is positioned on the caret while idle, so clicks must always + // hit the outer overlay (keeping offset coordinates stable); focus is given + // to the surface programmatically from the overlay's pointer-down. + pointer-events: none; +} + +// Positioned wrapper of the capture surface, translated during an IME +// composition so the candidate window follows the WASM caret. +.ime-anchor { + position: absolute; + top: 0; + left: 0; + width: 100%; + height: 100%; + pointer-events: none; } diff --git a/frontend/src/app/main/ui/workspace/sidebar/debug.cljs b/frontend/src/app/main/ui/workspace/sidebar/debug.cljs index 51435e5779..3a6ab4b60d 100644 --- a/frontend/src/app/main/ui/workspace/sidebar/debug.cljs +++ b/frontend/src/app/main/ui/workspace/sidebar/debug.cljs @@ -22,6 +22,7 @@ "Debug options that don't require a page reload to take effect. These options are handled reactively via okulary subscriptions." #{:shape-panel + :ime :show-ids :show-touched :components-debugger}) diff --git a/frontend/src/app/main/ui/workspace/viewport/selection.cljs b/frontend/src/app/main/ui/workspace/viewport/selection.cljs index a4c378fa8d..d064e717ec 100644 --- a/frontend/src/app/main/ui/workspace/viewport/selection.cljs +++ b/frontend/src/app/main/ui/workspace/viewport/selection.cljs @@ -31,6 +31,7 @@ [app.util.debug :as dbg] [app.util.dom :as dom] [app.util.object :as obj] + [app.util.text.writing-mode :as wm] [potok.v2.core :as ptk] [rumext.v2 :as mf])) @@ -563,23 +564,31 @@ (dom/stop-propagation event) (let [target (dom/get-current-target event) position (-> (dom/get-data target "position") - (keyword))] + (keyword)) + horizontal-resize? (contains? #{:right :left} position) + vertical-resize? (contains? #{:top :bottom} position) + vertical-text? (wm/vertical-text-content? (get shape :content)) + wrap-axis-resize? (if vertical-text? + vertical-resize? + horizontal-resize?) + resize-direction (if horizontal-resize? :horizontal :vertical)] (cond - ;; If text and in auto-width and the resize is horizontal, switch to auto-height and mark direction + ;; Resizing auto-width along the wrap axis (vertical under + ;; vertical writing) switches to auto-height. (and (= shape-type :text) (= grow-type :auto-width) - (or (= position :right) (= position :left))) - (st/emit! (dwsh/update-shapes [shape-id] #(-> % (assoc :grow-type :auto-height) (assoc :last-resize-direction :horizontal)))) - ;; If text and in auto-height and the resize is horizontal, mark direction but do not change grow-type + wrap-axis-resize?) + (st/emit! (dwsh/update-shapes [shape-id] + #(assoc % + :grow-type :auto-height + :last-resize-direction resize-direction))) + (and (= shape-type :text) (= grow-type :auto-height) - (or (= position :right) (= position :left))) - (st/emit! (dwsh/update-shapes [shape-id] #(assoc % :last-resize-direction :horizontal))) - ;; If text and in auto-height and the resize is vertical, mark direction - (and (= shape-type :text) - (= grow-type :auto-height) - (or (= position :top) (= position :bottom))) - (st/emit! (dwsh/update-shapes [shape-id] #(assoc % :last-resize-direction :vertical))) + (or horizontal-resize? vertical-resize?)) + (st/emit! (dwsh/update-shapes [shape-id] + #(assoc % :last-resize-direction resize-direction))) + :else nil) (st/emit! (dw/start-resize position #{shape-id} shape)))))) diff --git a/frontend/src/debug.cljs b/frontend/src/debug.cljs index 7d71812c19..f21423a6f1 100644 --- a/frontend/src/debug.cljs +++ b/frontend/src/debug.cljs @@ -88,7 +88,8 @@ (set! st/*debug-events-time* true) nil) - (js* "app.main.reinit()")) + (when-not (= option :ime) + (js* "app.main.reinit()"))) (defn disable! [option] @@ -101,7 +102,8 @@ (set! st/*debug-events-time* false) nil) - (js* "app.main.reinit()")) + (when-not (= option :ime) + (js* "app.main.reinit()"))) (defn ^:export toggle-debug [name] @@ -736,4 +738,4 @@ (dw/delete-shapes page-id ids)) unrelated-items-by-page) [(dwu/commit-undo-transaction undo-id)])))) - nil)) \ No newline at end of file + nil)) diff --git a/frontend/test/frontend_tests/data/workspace_texts_test.cljs b/frontend/test/frontend_tests/data/workspace_texts_test.cljs index eac8693cff..67e840fbde 100644 --- a/frontend/test/frontend_tests/data/workspace_texts_test.cljs +++ b/frontend/test/frontend_tests/data/workspace_texts_test.cljs @@ -6,6 +6,7 @@ (ns frontend-tests.data.workspace-texts-test (:require + [app.common.geom.point :as gpt] [app.common.geom.rect :as grc] [app.common.test-helpers.files :as cthf] [app.common.test-helpers.shapes :as cths] @@ -13,11 +14,104 @@ [app.common.types.shape :as cts] [app.common.types.text :as txt] [app.common.uuid :as uuid] + [app.main.data.workspace.modifiers :as dwm] + [app.main.data.workspace.shapes :as dwsh] [app.main.data.workspace.texts :as dwt] [app.main.data.workspace.texts-events :as dwte] + [app.main.data.workspace.wasm-text :as dwwt] + [app.main.ui.shapes.text.styles :as text.styles] [app.main.ui.workspace.shapes.text.viewport-texts-html :as vth] + [beicon.v2.core :as rx] [cljs.test :as t :include-macros true] - [frontend-tests.helpers.state :as ths])) + [frontend-tests.helpers.state :as ths] + [potok.v2.core :as ptk])) + +(defn- text-content + [writing-mode] + {:children [{:children [{:writing-mode writing-mode}]}]}) + +(t/deftest vertical-auto-height-grows-width + (let [selrect {:width 100 :height 200} + dimension {:width 240 :height 360} + content (text-content "vertical-rl")] + (t/is (= {:width 240 :height 200} + (dwwt/resolve-text-size selrect :auto-height content dimension))))) + +(t/deftest horizontal-auto-height-grows-height + (let [selrect {:width 100 :height 200} + dimension {:width 240 :height 360} + content (text-content "horizontal-tb")] + (t/is (= {:width 100 :height 360} + (dwwt/resolve-text-size selrect :auto-height content dimension))))) + +(t/deftest vertical-grow-type-resize-axes-are-remapped + (t/is (= :auto-height + (dwm/next-grow-type :auto-width (gpt/point 1 2) true))) + (t/is (= :fixed + (dwm/next-grow-type :auto-height (gpt/point 2 1) true))) + (t/is (= :auto-height + (dwm/next-grow-type :auto-height (gpt/point 1 2) true)))) + +(t/deftest vertical-export-styles-enable-inter-script-spacing + (let [vertical (text.styles/generate-paragraph-styles + nil + {:writing-mode "vertical-rl" + :text-orientation "upright"}) + horizontal (text.styles/generate-paragraph-styles + nil + {:writing-mode "horizontal-tb"})] + (t/is (= "vertical-rl" (aget vertical "writingMode"))) + (t/is (= "upright" (aget vertical "textOrientation"))) + (t/is (= "normal" (aget vertical "textAutospace"))) + (t/is (nil? (aget horizontal "textAutospace"))))) + +(t/deftest text-export-styles-emit-font-features + (let [palt (text.styles/generate-text-styles + {:grow-type :fixed} + {:font-features "palt" + :font-size "20" + :fills [{:fill-color "#000000" :fill-opacity 1}]}) + none (text.styles/generate-text-styles + {:grow-type :fixed} + {:font-features "none" + :font-size "20" + :fills [{:fill-color "#000000" :fill-opacity 1}]})] + (t/is (= "\"palt\"" (aget palt "fontFeatureSettings"))) + (t/is (nil? (aget none "fontFeatureSettings"))))) + +(t/deftest text-export-styles-emit-annotation-clearance + (let [style (text.styles/generate-text-styles + {:grow-type :fixed} + {:annotation-clearance "auto" + :line-height "1.2" + :ruby "かんじ" + :text-emphasis "filled-dot" + :font-size "20" + :fills [{:fill-color "#000000" :fill-opacity 1}]})] + (t/is (nil? (aget style "--annotation-clearance"))) + (t/is (= "calc(max(var(--paragraph-line-height, 1.2), 1.2) + 1)" + (aget style "lineHeight"))))) + +(t/deftest annotation-clearance-adds-the-ruby-size-to-the-paragraph-line-height + (let [style (text.styles/generate-text-styles + {:grow-type :fixed} + {:annotation-clearance "auto" + :ruby "かんじ" + :ruby-size "quarter" + :font-size "20" + :fills [{:fill-color "#000000" :fill-opacity 1}]}) + paragraph (text.styles/generate-paragraph-styles nil {:line-height "2"})] + (t/is (= "calc(var(--paragraph-line-height, 1.2) + 0.25)" (aget style "lineHeight"))) + (t/is (= "2" (aget paragraph "--paragraph-line-height"))))) + +(t/deftest annotation-clearance-none-keeps-the-line-height + (let [style (text.styles/generate-text-styles + {:grow-type :fixed} + {:annotation-clearance "none" + :ruby "かんじ" + :font-size "20" + :fills [{:fill-color "#000000" :fill-opacity 1}]})] + (t/is (nil? (aget style "lineHeight"))))) ;; --------------------------------------------------------------------------- ;; Helpers @@ -656,3 +750,79 @@ (assoc data :text-geometry {bottom-id {:origins [[origin]]}})))))) (t/testing "a copy without geometry gives a payload without it" (t/is (not (contains? (#'dwt/copied-shapes->payload data) :geometry)))))) + +(defn- spans-content + [& spans] + {:type "root" + :children [{:type "paragraph-set" + :children [{:type "paragraph" + :children (vec spans)}]}]}) + +(defn- content-spans + [shape] + (txt/node-seq txt/is-text-node? (:content shape))) + +(t/deftest range-style-over-part-of-a-ruby-span-styles-the-whole-span + (let [shape {:type :text + :content (spans-content {:text "あ" :font-size "14"} + {:text "漢字" :font-size "14" :ruby "かんじ"} + {:text "い" :font-size "14"})} + result (dwt/update-text-range-attrs shape 1 2 {:font-size "30"})] + (t/is (= [["あ" "14" nil] ["漢字" "30" "かんじ"] ["い" "14" nil]] + (mapv (juxt :text :font-size :ruby) (content-spans result)))))) + +(t/deftest range-style-over-part-of-a-warichu-span-styles-the-whole-span + (let [shape {:type :text + :content (spans-content {:text "本文" :font-size "14"} + {:text "割注" :font-size "14" :warichu "warichu"})} + result (dwt/update-text-range-attrs shape 0 3 {:font-size "30"})] + (t/is (= [["本文" "30" nil] ["割注" "30" "warichu"]] + (mapv (juxt :text :font-size :warichu) (content-spans result)))))) + +(t/deftest single-span-range-is-false-across-span-boundaries + (let [content (spans-content {:text "漢" :font-weight "700"} + {:text "字かな" :font-weight "400"})] + (t/is (true? (dwt/single-span-range? content 1 3))) + (t/is (false? (dwt/single-span-range? content 0 2))))) + +(defn- paragraphs-content + [& texts] + {:type "root" + :children [{:type "paragraph-set" + :children (mapv (fn [text] + {:type "paragraph" + :children [{:text text :font-size "14"}]}) + texts)}]}) + +(t/deftest range-style-in-a-later-paragraph-uses-document-offsets + (let [shape {:type :text :content (paragraphs-content "東京" "京都")} + result (dwt/update-text-range-attrs shape 3 5 {:font-size "30"})] + (t/is (= [["東京" "14"] ["京都" "30"]] + (mapv (juxt :text :font-size) (content-spans result)))))) + +(t/deftest range-style-in-the-first-paragraph-leaves-later-paragraphs-alone + (let [shape {:type :text :content (paragraphs-content "東京" "京都")} + result (dwt/update-text-range-attrs shape 0 2 {:font-size "30"})] + (t/is (= [["東京" "30"] ["京都" "14"]] + (mapv (juxt :text :font-size) (content-spans result)))))) + +(t/deftest single-span-range-counts-each-paragraph-at-its-own-offset + (let [content (paragraphs-content "東京" "京都")] + (t/is (true? (dwt/single-span-range? content 0 2))) + (t/is (true? (dwt/single-span-range? content 3 5))) + (t/is (false? (dwt/single-span-range? content 1 4))))) + +(t/deftest ruby-presentation-of-several-texts-is-one-shape-update + (let [file-id (uuid/next) + page-id (uuid/next) + ids [(uuid/next) (uuid/next) (uuid/next)] + objects (into {} (map (fn [id] [id {:id id :type :text}])) ids) + state {:current-file-id file-id + :current-page-id page-id + :features #{} + :files {file-id {:data {:pages-index {page-id {:objects objects}}}}}} + events (atom [])] + (->> (ptk/watch (dwt/update-all-ruby-presentation ids {:ruby-size "third"}) state nil) + (rx/subs! #(swap! events conj %))) + (let [updates (filter #(= ::dwsh/update-shapes (ptk/type %)) @events)] + (t/is (= 1 (count updates)))))) diff --git a/frontend/test/frontend_tests/render_wasm/texts_test.cljs b/frontend/test/frontend_tests/render_wasm/texts_test.cljs new file mode 100644 index 0000000000..de42a8d277 --- /dev/null +++ b/frontend/test/frontend_tests/render_wasm/texts_test.cljs @@ -0,0 +1,450 @@ +;; This Source Code Form is subject to the terms of the Mozilla Public +;; License, v. 2.0. If a copy of the MPL was not distributed with this +;; file, You can obtain one at http://mozilla.org/MPL/2.0/. +;; +;; Copyright (c) KALEIDOS INC Sucursal en España SL + +(ns frontend-tests.render-wasm.texts-test + "Japanese text support in the WASM bridge: span serialization, editor + selection styles, IME composition caret placement, and the CJK script + classification used to pick Noto fallback fonts." + (:require + [app.common.fonts :as cfnt] + [app.common.render-wasm.serializers :as sr] + [app.common.render-wasm.text-content :as tc] + [app.common.render-wasm.wasm :as wasm] + [app.common.types.text.cjk-layout :as cjk] + [app.config :as cfg] + [app.main.ui.workspace.shapes.text.ime-debug :as ime-debug] + [app.main.ui.workspace.shapes.text.v3-editor :as v3-editor] + [app.render-wasm.api :as api] + [app.render-wasm.api.texts :as texts] + [app.render-wasm.text-editor :as text-editor] + [cljs.test :as t :include-macros true] + [frontend-tests.helpers.mock :as mock])) + +(defn- write-spans + [offset dview spans paragraph] + (@#'tc/write-spans offset dview spans paragraph cfnt/font-id->uuid)) + +(defn- langs [text] + (cfnt/collect-used-languages #{} text)) + +(defn- font-ids + [fonts] + (into #{} (map :font-id) fonts)) + +(defn- editor-content + [& paragraphs] + {:type "root" + :children [{:type "paragraph-set" + :children (mapv (fn [spans] + {:type "paragraph" + :children spans}) + paragraphs)}]}) + +(defn- selection + [anchor-para anchor-offset focus-para focus-offset] + {:anchor-para anchor-para + :anchor-offset anchor-offset + :focus-para focus-para + :focus-offset focus-offset}) + +(t/deftest japanese-styles-follow-the-selected-span + (let [content (editor-content + [{:text "日" :ruby "にち" :text-emphasis "filled-dot"} + {:text "本"}])] + (t/is (= {:text-combine-upright "none" + :text-emphasis "filled-dot" + :ruby "にち" + :ruby-hidden false + :ruby-size "half" + :ruby-align "space-around" + :ruby-overhang "auto" + :ruby-side "over" + :warichu "none" + :font-features "none" + :annotation-clearance "none"} + (text-editor/selection-japanese-styles + content (selection 0 0 0 1)))) + (t/is (= {:text-combine-upright "none" + :text-emphasis "none" + :ruby "" + :ruby-hidden false + :ruby-size "half" + :ruby-align "space-around" + :ruby-overhang "auto" + :ruby-side "over" + :warichu "none" + :font-features "none" + :annotation-clearance "none"} + (text-editor/selection-japanese-styles + content (selection 0 1 0 2)))))) + +(t/deftest japanese-styles-report-mixed-ranges-and-follow-the-caret + (let [content (editor-content + [{:text "平成" + :text-combine-upright "digits2" + :text-emphasis "filled-dot" + :ruby "へいせい" + :ruby-hidden true + :ruby-size "third" + :ruby-align "center" + :ruby-overhang "none" + :ruby-side "under" + :annotation-clearance "auto"} + {:text "年" + :ruby-hidden false + :warichu "warichu" + :font-features "vpal"}])] + (t/is (= {:text-combine-upright :multiple + :text-emphasis :multiple + :ruby :multiple + :ruby-hidden :multiple + :ruby-size :multiple + :ruby-align :multiple + :ruby-overhang :multiple + :ruby-side :multiple + :warichu :multiple + :font-features :multiple + :annotation-clearance :multiple} + (text-editor/selection-japanese-styles + content (selection 0 0 0 3)))) + ;; A caret at the boundary belongs to the preceding span, as in WASM. + (t/is (= "digits2" + (:text-combine-upright + (text-editor/selection-japanese-styles + content (selection 0 2 0 2))))) + (t/is (= "warichu" + (:warichu + (text-editor/selection-japanese-styles + content (selection 0 3 0 3))))))) + +(t/deftest japanese-selection-offsets-are-utf16-code-units + (let [content (editor-content + [{:text "😀"} + {:text "日" :ruby "にち"}])] + (t/is (= "にち" + (:ruby + (text-editor/selection-japanese-styles + content (selection 0 2 0 3))))))) + +(t/deftest composition-caret-goes-to-the-end-of-the-changed-part + ;; Typing appends, converting replaces all, and a candidate for a middle + ;; clause changes only that clause. + (t/is (= 1 (v3-editor/changed-span-end "" "に"))) + (t/is (= 2 (v3-editor/changed-span-end "に" "にほ"))) + (t/is (= 3 (v3-editor/changed-span-end "にほんご" "日本語"))) + (t/is (= 5 (v3-editor/changed-span-end "日本語を書く" "日本語を掻く"))) + (t/is (= 3 (v3-editor/changed-span-end "ああ" "あああ"))) + (t/is (= 0 (v3-editor/changed-span-end "にほ" ""))) + (t/is (= 2 (v3-editor/changed-span-end "😀" "😀😀")))) + +(t/deftest ime-debug-caret-bounds-follow-viewport-zoom-and-pan + (t/is (= {:x 210 :y 380 :width 48 :height 2} + (@#'ime-debug/screen-rect + {:x 100 :y 200 :width 24 :height 1} + #js {:a 2 :b 0 :c 0 :d 2 :e 10 :f -20})))) + +(t/deftest ime-debug-caret-bounds-follow-shape-rotation + (t/is (= {:x 119 :y 140 :width 1 :height 24} + (@#'ime-debug/screen-rect + {:x 100 :y 200 :width 24 :height 1} + #js {:a 0 :b 1 :c -1 :d 0 :e 320 :f 40})))) + +(t/deftest ime-debug-composition-offsets-use-dom-utf16-units + (let [prefix #js {:length 3} + kana #js {:length 2} + suffix #js {:length 1} + nodes [prefix kana suffix]] + ;; A prefix such as "a😀" takes three DOM units, not two code points. + ;; The range must exclude zero-length rectangles from adjacent text nodes. + (t/is (= [kana 0] (@#'ime-debug/text-position nodes 3 :start))) + (t/is (= [kana 2] (@#'ime-debug/text-position nodes 5 :end))) + (t/is (nil? (@#'ime-debug/text-position nodes 7 :end))))) + +(defn- ime-surface-style + ([platform vertical? rect] + (ime-surface-style platform vertical? rect 1)) + ([platform vertical? rect zoom] + (ime-surface-style platform vertical? rect zoom :chrome)) + ([platform vertical? rect zoom browser] + (mock/with-mocks* + {cfg/browser browser + cfg/check-platform? (fn [candidate] (= platform candidate)) + text-editor/text-editor-get-cursor-rect (fn [] rect)} + (let [node #js {:style #js {} + :closest (fn [_] #js {:getScreenCTM (fn [] #js {:c 0 :d zoom})})} + wrapper #js {:style #js {:transform "translate(10px, 20px)" + :left "-36px" :top "72px"}} + origin {:x 100 :y 80 :width 900 :vertical? vertical?} + anchor (@#'v3-editor/update-ime-caret! node wrapper origin)] + {:left (+ (:x origin) (js/parseFloat (.. node -style -left))) + :right (+ (:x origin) (:width origin) (js/parseFloat (.. node -style -left))) + :top (+ (:y origin) (js/parseFloat (.. node -style -top))) + :writing-mode (.. node -style -writingMode) + :transform (.. node -style -transform) + :overflow (.. node -style -overflow) + :padding (@#'v3-editor/ime-overlay-padding vertical? 900) + :anchor anchor + :wrapper-left (.. wrapper -style -left) + :wrapper-top (.. wrapper -style -top) + :wrapper-transform (.. wrapper -style -transform)})))) + +(t/deftest ^:async macos-vertical-ime-anchor-matches-the-text-column + (doseq [width [12 24 72] + browser [:chrome :firefox :edge]] + (let [rect {:x 400 :y 180 :width width :height 1} + result (await (ime-surface-style :macos true rect 1 browser))] + ;; vertical-rl puts the caret cell immediately left of the surface's + ;; right edge. Match the WASM column without a horizontal offset. + (t/is (= (+ (:x rect) (:width rect)) (:right result))) + (t/is (= (:y rect) (:top result))) + (t/is (= "vertical-rl" (:writing-mode result))) + (t/is (= {:left 900 :right 0} (:padding result))) + (t/is (= "clip" (:overflow result))) + (t/is (true? (:vertical? (:anchor result)))) + (t/is (= "" (:wrapper-left result))) + (t/is (= "" (:wrapper-top result))) + (t/is (= "" (:wrapper-transform result)))))) + +(t/deftest ^:async macos-safari-vertical-ime-anchor-clears-the-text-column + ;; Safari's native candidate window is horizontal even for vertical text. + ;; Its anchor needs room to the right, including at the text shape's edge. + (doseq [browser [:safari :safari-16 :safari-17 :safari-18 :safari-26] + width [12 24 72]] + (let [rect {:x 400 :y 180 :width width :height 1} + result (await (ime-surface-style :macos true rect 1 browser))] + (t/is (> (- (:right result) width) (+ (:x rect) width))) + (t/is (= (:y rect) (:top result))) + (t/is (= {:left 900 :right 900} (:padding result))) + (t/is (= "vertical-rl" (:writing-mode result))) + (t/is (= "clip" (:overflow result)))))) + +(t/deftest ^:async macos-vertical-ime-character-bounds-stay-nonempty + ;; Chromium derives a vertical character rectangle from integer caret y + ;; positions. Subpixel advances become empty rectangles, which macOS uses + ;; to position its candidate window. Exercise long compositions and zoom. + (doseq [width [12 24 72] + zoom [0.25 1 4] + browser [:chrome :firefox :safari-26]] + (let [rect {:x 400 :y 180 :width width :height 1} + result (await (ime-surface-style :macos true rect zoom browser)) + scale (js/parseFloat (subs (:transform result) 7)) + step (* width scale zoom) + ys (map #(js/Math.floor (+ 180 (* % step))) (range 81))] + (t/is (every? pos? (map - (rest ys) ys)))))) + +(t/deftest ^:async linux-and-windows-vertical-ime-anchor-stays-right-of-the-text-column + (doseq [platform [:linux :windows] + browser [:chrome :firefox :safari]] + (let [rect {:x 400 :y 180 :width 24 :height 1} + result (await (ime-surface-style platform true rect 1 browser))] + ;; vertical-rl places the DOM caret in the rightmost cell of the surface. + (t/is (> (- (:right result) (:width rect)) (+ (:x rect) (:width rect)))) + (t/is (= (:y rect) (:top result))) + (t/is (= {:left 0 :right 900} (:padding result))) + (t/is (= "vertical-rl" (:writing-mode result))) + (t/is (= "scaleY(0.001)" (:transform result))) + (t/is (= "" (:overflow result)))))) + +(t/deftest ^:async horizontal-ime-anchor-stays-on-the-caret + (doseq [platform [:macos :linux :windows] + browser [:chrome :firefox :safari]] + (let [rect {:x 400 :y 180 :width 1 :height 24} + result (await (ime-surface-style platform false rect 1 browser))] + (t/is (= (:x rect) (:left result))) + (t/is (= (:y rect) (:top result))) + (t/is (= {:left 0 :right 0} (:padding result))) + (t/is (= "" (:writing-mode result))) + (t/is (= "scaleX(0.001)" (:transform result)))))) + +(t/deftest ^:async macos-vertical-ime-follows-the-caret-in-layout + (let [anchor {:x 400 :y 180 :width 24 :height 1 :vertical? true} + current (atom {:x 400 :y 252 :width 24 :height 1}) + wrapper #js {:style #js {:transform ""}}] + (await + (mock/with-mocks* + {cfg/check-platform? (fn [candidate] (= candidate :macos)) + text-editor/text-editor-get-cursor-rect (fn [] @current)} + (@#'v3-editor/follow-ime-caret! wrapper anchor) + (t/is (= "72px" (.. wrapper -style -top))) + (t/is (= "0px" (.. wrapper -style -left))) + (t/is (= "" (.. wrapper -style -transform))) + ;; Moving into another vertical column resets the inline position. + (reset! current {:x 364 :y 180 :width 24 :height 1}) + (@#'v3-editor/follow-ime-caret! wrapper anchor) + (t/is (= "0px" (.. wrapper -style -top))) + (t/is (= "-36px" (.. wrapper -style -left))))))) + +(t/deftest ^:async other-ime-directions-keep-following-with-a-transform + (doseq [[platform vertical?] [[:linux true] [:linux false] [:macos false] + [:windows true] [:windows false]]] + (let [anchor {:x 400 :y 180 :vertical? vertical?} + wrapper #js {:style #js {}}] + (await + (mock/with-mocks* + {cfg/check-platform? (fn [candidate] (= candidate platform)) + text-editor/text-editor-get-cursor-rect (fn [] {:x 424 :y 252})} + (@#'v3-editor/follow-ime-caret! wrapper anchor) + (t/is (= "translate(24px, 72px)" (.. wrapper -style -transform)))))))) + +(t/deftest whole-shape-paragraph-attrs-apply-to-every-paragraph + (let [content (-> (editor-content [{:text "一"}] [{:text "二"}] [{:text "三"}]) + (assoc-in [:children 0 :children 2 :text-orientation] "upright")) + result (text-editor/apply-paragraph-attrs-to-range + content + (selection 1 0 1 0) + {:writing-mode "vertical-rl" + :text-orientation nil + :text-align "center"}) + paragraphs (-> result :children first :children)] + (t/is (every? #(= "vertical-rl" (:writing-mode %)) paragraphs)) + (t/is (not-any? #(contains? % :text-orientation) paragraphs)) + (t/is (= [nil "center" nil] (mapv :text-align paragraphs))))) + +;; Byte offsets inside one span's attribute block (see `write-spans` and +;; RawTextSpan in render-wasm). +(def ^:private span-attr-offset + {:text-combine-upright 5 + :font-features 8 + :annotation-clearance 9 + :ruby-size 10 + :ruby-align 11 + :ruby-overhang 12 + :ruby-side 13 + :padding [14 15] + :ruby-length 72}) + +(def ^:private span-paragraph + {:font-size "16" :font-weight "400" :line-height "1"}) + +(defn- serialized-span + "DataView over the attribute block `write-spans` produces for `span`." + [span] + (let [dview (js/DataView. (js/ArrayBuffer. 256))] + (write-spans 0 dview [(merge {:font-size "16" :font-weight "400"} span)] span-paragraph) + dview)) + +(defn- span-byte + [dview attr] + (.getUint8 dview (get span-attr-offset attr))) + +(defn- padding-bytes + [dview] + (mapv #(.getUint8 dview %) (:padding span-attr-offset))) + +(t/deftest write-spans-serializes-font-features + (let [dview (serialized-span {:text "日本語" :font-features "vpal"})] + (t/is (= 2 (span-byte dview :font-features))) + (t/is (= 0 (span-byte dview :annotation-clearance))) + (t/is (= [0 0] (padding-bytes dview))))) + +(t/deftest write-spans-serializes-annotation-clearance + (let [dview (serialized-span {:text "漢字" :annotation-clearance "auto"})] + (t/is (= 1 (span-byte dview :annotation-clearance))) + (t/is (= [0 0] (padding-bytes dview))))) + +(t/deftest write-spans-serializes-ruby-customization + (let [dview (serialized-span {:text "漢字" + :ruby-size "quarter" + :ruby-align "space-between" + :ruby-overhang "none" + :ruby-side "under"})] + (t/is (= [2 3 1 1] + (mapv #(span-byte dview %) [:ruby-size :ruby-align :ruby-overhang :ruby-side]))) + (t/is (= [0 0] (padding-bytes dview))))) + +(t/deftest write-spans-omits-hidden-ruby + (let [ruby-length #(.getInt32 % (:ruby-length span-attr-offset) true)] + (t/is (= 0 (ruby-length (serialized-span {:text "日" :ruby "にち" :ruby-hidden true})))) + (t/is (pos? (ruby-length (serialized-span {:text "日" :ruby "にち" :ruby-hidden false})))) + ;; A stored nil reading sends no bytes, as an absent one does. + (t/is (= 0 (ruby-length (serialized-span {:text "日" :ruby nil})))))) + +(t/deftest write-spans-serializes-counted-digits-tcy + (let [dview (serialized-span {:text "平成31年" :text-combine-upright "digits2"})] + (t/is (= 3 (span-byte dview :text-combine-upright))))) + +(t/deftest ruby-text-participates-in-live-and-reload-fallback-discovery + (let [content {:children + [{:children + [{:children + [{:text "Penpot" + :ruby "ぺんぽっと😀"}]}]}]} + expected #{"gfont-noto-sans-jp" "gfont-noto-color-emoji"}] + (with-redefs [texts/write-shape-text (fn [& _])] + (t/is (every? (font-ids (api/fonts-from-text-content content true)) expected))) + (t/is (every? (font-ids (api/fonts-from-text-content content false)) expected)))) + +(t/deftest classification-kana + ;; Hiragana/katakana are unambiguously Japanese. + (t/is (= #{:japanese} (langs "ひらがなとカタカナ"))) + ;; Half-width katakana too. + (t/is (= #{:japanese} (langs "デザイン")))) + +(t/deftest classification-han-is-ambiguous + ;; Kanji-only text is ambiguous Han, not a concrete language. + (t/is (= #{:han} (langs "東京都渋谷区神南一丁目")))) + +(t/deftest classification-cjk-punctuation + ;; CJK punctuation and full-width forms match the shared class. + (t/is (= #{:cjk-punctuation} (langs "、。「」『』()"))) + (t/is (= #{:cjk-punctuation} (langs "!?:;123ABC")))) + +(t/deftest classification-mixed-japanese + (t/is (= #{:japanese :han :cjk-punctuation} + (langs "「こんにちは」と彼は言った。")))) + +(t/deftest classification-korean + (t/is (= #{:korean} (langs "안녕하세요")))) + +(t/deftest resolve-kana-wins-over-locale + ;; Kana in the same content implies Japanese regardless of locale. + (t/is (= #{:japanese} + (cfnt/resolve-ambiguous-cjk #{:japanese :han :cjk-punctuation} "zh"))) + (t/is (= #{:japanese} + (cfnt/resolve-ambiguous-cjk #{:japanese :han} "en")))) + +(t/deftest resolve-hangul-wins-over-locale + (t/is (= #{:korean} + (cfnt/resolve-ambiguous-cjk #{:korean :han} "ja")))) + +(t/deftest resolve-han-only-uses-locale + (t/is (= #{:japanese} (cfnt/resolve-ambiguous-cjk #{:han} "ja"))) + (t/is (= #{:japanese} (cfnt/resolve-ambiguous-cjk #{:han} "ja_paid"))) + (t/is (= #{:korean} (cfnt/resolve-ambiguous-cjk #{:han} "ko"))) + (t/is (= #{:chinese} (cfnt/resolve-ambiguous-cjk #{:han} "zh_cn")))) + +(t/deftest resolve-han-only-defaults-to-chinese + ;; Without kana, hangul or a CJK locale, Han resolves to Chinese (Noto Sans SC). + (t/is (= #{:chinese} (cfnt/resolve-ambiguous-cjk #{:han} "en"))) + (t/is (= #{:chinese} (cfnt/resolve-ambiguous-cjk #{:han} nil)))) + +(t/deftest resolve-punctuation-only + (t/is (= #{:japanese} (cfnt/resolve-ambiguous-cjk #{:cjk-punctuation} "ja"))) + (t/is (= #{:chinese} (cfnt/resolve-ambiguous-cjk #{:cjk-punctuation} "en")))) + +(t/deftest resolve-leaves-unambiguous-sets-alone + (t/is (= #{:latin-ext} (cfnt/resolve-ambiguous-cjk #{:latin-ext} "ja"))) + (t/is (= #{} (cfnt/resolve-ambiguous-cjk #{} "ja"))) + ;; Other detected languages survive resolution. + (t/is (= #{:japanese :cyrillic} + (cfnt/resolve-ambiguous-cjk #{:han :cyrillic} "ja")))) + +(t/deftest resolve-han-unification + ;; A kanji-only Japanese address resolves to Japanese under a ja locale. + (t/is (= #{:japanese} + (cfnt/resolve-ambiguous-cjk (langs "東京都渋谷区神南一丁目") "ja")))) + +(t/deftest wasm-enum-exports-match-the-japanese-value-table + (doseq [[attr values] cjk/enum-values] + (let [exported (js->clj (unchecked-get wasm/serializers (name attr)))] + (t/is (= values (mapv key (sort-by val exported))) (name attr))))) + +(t/deftest unset-or-unknown-japanese-values-serialize-as-the-default + (t/is (= 0 (sr/translate-text-layout-enum :ruby-side nil))) + (t/is (= 0 (sr/translate-text-layout-enum :ruby-side ""))) + (t/is (= 0 (sr/translate-text-layout-enum :ruby-side "left"))) + (t/is (= 1 (sr/translate-text-layout-enum :ruby-side "under"))) + (t/is (= 1 (sr/translate-text-layout-enum :ruby-side :under)))) diff --git a/frontend/test/frontend_tests/runner.cljs b/frontend/test/frontend_tests/runner.cljs index 5e18876e88..f68303e117 100644 --- a/frontend/test/frontend_tests/runner.cljs +++ b/frontend/test/frontend_tests/runner.cljs @@ -95,6 +95,7 @@ [frontend-tests.render-wasm.text-editor-content-range-test] [frontend-tests.render-wasm.text-paste-test] [frontend-tests.render-wasm.webgl-test] + [frontend-tests.render-wasm.texts-test] [frontend-tests.router-test] [frontend-tests.svg-fills-test] [frontend-tests.svg-filters-test] @@ -119,6 +120,7 @@ [frontend-tests.ui.comments-clustering-test] [frontend-tests.ui.comments-position-modifier-test] [frontend-tests.ui.dashboard-import-linking-test] + [frontend-tests.ui.css-cursors-test] [frontend-tests.ui.drawarea-test] [frontend-tests.ui.ds-controls-numeric-input-test] [frontend-tests.ui.error-boundary-test] @@ -259,6 +261,7 @@ 'frontend-tests.render-wasm.text-editor-content-range-test 'frontend-tests.render-wasm.text-paste-test 'frontend-tests.render-wasm.webgl-test + 'frontend-tests.render-wasm.texts-test 'frontend-tests.browser-history-test 'frontend-tests.router-test 'frontend-tests.svg-fills-test @@ -283,6 +286,7 @@ 'frontend-tests.ui.comments-clustering-test 'frontend-tests.ui.comments-position-modifier-test 'frontend-tests.ui.dashboard-import-linking-test + 'frontend-tests.ui.css-cursors-test 'frontend-tests.ui.drawarea-test 'frontend-tests.ui.ds-controls-numeric-input-test 'frontend-tests.ui.error-boundary-test diff --git a/frontend/test/frontend_tests/ui/css_cursors_test.cljs b/frontend/test/frontend_tests/ui/css_cursors_test.cljs new file mode 100644 index 0000000000..5478bca52c --- /dev/null +++ b/frontend/test/frontend_tests/ui/css_cursors_test.cljs @@ -0,0 +1,9 @@ +(ns frontend-tests.ui.css-cursors-test + (:require + [app.main.ui.css-cursors :as cursors] + [cljs.test :as t])) + +(t/deftest text-cursor-follows-writing-direction-and-shape-rotation + (t/is (= "cursor-text-0" (cursors/get-text 0 false))) + (t/is (= "cursor-text-90" (cursors/get-text 0 true))) + (t/is (= "cursor-text-10" (cursors/get-text 280 true))))