CommonMark forbids an info string on a closing fence, so "```python" inside a
plain ``` fence is content, not the end of the block. The closer check that two
fence consumers share only compared character and run length, which ends the
block early and pushes whatever follows back into prose: on the render path a
system marker inside the sample loses its tags and its content becomes visible
text, and on the export path stripInternalMarkers deletes real user content from
the thread markdown.
mermaid.ts already had the correct predicate privately. Move it into the shared
leaf module and use it from both consumers, so the three fence scanners in
core/streamdown and core/messages agree on what closes a block.
* feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash
Implements docs/superpowers/specs/2026-09-12-projects-mvp-phase2-design.md
(issue #5160, tracker #5129) in the slice order of the spec's §16.
Slices:
- A: ProjectsConfig + write-time 422 UTF-8 byte cap; PROJECT_CONTEXT_KEY
admission pinning (both server-owned sets + worker hoist); latest-only
request-scoped <project> block via DynamicContextMiddleware
wrap_model_call/awrap_model_call (idempotent reassembly, reserved ID
prefix + marker + provenance, never persisted); journal audit
fingerprints; Instructions tab.
- B: ProjectDocumentRow + migration 0023; ProjectDocumentRepository with
locked check-and-set; hash-qualified immutable shelf storage with
Paths helpers; upload/list/content/delete-to-trash routes; project
delete trashes the shelf in-transaction; request-scoped bounded
<documents> index with honest count/shown + actionable overflow note;
list_project_documents/read_project_document tools registered only on
pinned runs; PAT allowlist + drift guards; blocking-IO anchors.
- C: shared thread-upload ingestion service (uploads router refactored to
parity); POST from-thread with provenance; attach-to-thread with
lock-staged copy (archived source allowed); read-only thread-files
view with per-group truncation reporting.
- D: restore (restored/merged/not_found/no_target/content_missing; no
file moves), purge (continuous row lock across unlink/delete/commit,
retryable on FS errors), retention sweep (lazy + startup, 24h orphan
guard, row-side reconciliation never deletes).
- E: Documents tab (shelf + conversation-files browser, provenance,
archived banner, content-missing rows), /workspace/trash route,
sidebar entry, composer attach handoff, i18n (en-US/zh-CN), e2e mocks
+ specs.
Review hardening folded in (10 rounds, all with tests):
- force active shelf content (HTML/XML family) to download; nosniff on
artifact + content responses; unified unsandboxed-iframe PDF preview
(fixes the pre-existing Chromium sandbox blank in the artifact viewer)
- scope document trash to the URL project under the document lock
- atomic no-overwrite filename reservation for ALL ingestion (seeded
claims + os.link commit with suffix retry; same-name re-upload now
unique-names instead of replacing); hidden staging only, no visible
placeholders; lease cleanup on setup failure
- serialize conversion under the document lock with post-lock active
revalidation; drain locked filesystem work on cancellation; preserve
bytes when an insert's commit state is uncertain (including trashed
rows)
- original-integrity checks before serving text or cached conversions;
content_missing surfaced in list responses (UI reads the flag, no
409-probe); downloads always serve original bytes
- bounded streaming document reads with cached char counts; shelf limits
declared in middleware release identity
- thread-root confinement for from-thread sources; config fallback
rejects fractional/infinite values; composer counts staged
attachments; pending attachments persist until submission or removal;
in-flight instruction/rename edits survive save refetches; shelf and
trash pagination; conversation-file and thread-files pages stay
subscribed to refetches
Docs: README/README_zh, backend API.md/ARCHITECTURE.md, AGENTS.md
contracts, config.example.yaml projects block.
Review follow-ups (head b4807477 → this revision):
- The trash retention sweep is split so repeated lazy triggers stay
bounded: the indexed expiry purge still runs on every trigger
(GET /api/trash/documents, POST /api/trash/purge) while the
O(all rows + all files) reconciliation is throttled to one run per
user per 15 minutes (process-local, per-user window). The startup
sweep now runs as a background task instead of blocking gateway
readiness, and shutdown awaits it (bounded).
- The export scrub (stripInternalMarkers) is fence- and indentation-aware
like the render path, so a pasted, fenced <project>/<documents> snippet
survives markdown export while real injected blocks (never fenced) are
still removed. Fence regexes moved to a dependency-free leaf module to
avoid the messages↔streamdown import cycle.
- The artifact viewer's PDF iframe no longer carries an added title
attribute (the upstream e2e contract locates it via :not([title])), and
the upstream artifact-preview spec now pins the new contract: PDFs
render unsandboxed, images keep sandbox="".
* fix(projects): round-2 review — cancel an overrun trash sweep, restore the PDF frame title
- Shutdown cancelled only the shield around the background startup sweep,
so an all-users reconciliation that outlived the 5s budget kept walking
rows and files while the document repo and DB engine were disposed
underneath it. The wait now lives in `_shutdown_startup_trash_sweep`,
which cancels the task and drains it before worker exit: the shield
keeps the wait bounded, the cancel makes it final (CancelledError lands
at the sweep's next await, and `_run_startup_trash_sweep` only catches
`Exception`, so nothing swallows it).
- The browser-preview iframe lost `title={getFileName(filepath)}` in the
previous fix round, leaving the PDF frame without an accessible name
while its siblings keep theirs. Restore it (WCAG frame titles), assert
it in the DOM test, and anchor the e2e on `iframe[title="report.pdf"]`
instead of `iframe:not([title])`.
* fix(projects): round-3 review — report the sweep's late finish, not a phantom cancel
`Task.cancel()` returns False when the sweep already finished inside the
window between the deadline firing and the cancel, so the shutdown log
claimed a cancellation that never happened. Branch on that outcome: the
warning stays for a real cancel, a late finish is logged at info, and both
paths still reap the task before worker exit.
* fix(projects): round-4 review — make Empty trash delete what it confirms
`POST /api/trash/purge` only ran the retention sweep, and the sweep's
candidate selection is age-gated, so a freshly trashed document survived
"Empty trash" even though the confirmation promises that every listed
document is permanently deleted. With one trashed row the route answered
`{"purged": 0}` and left it in place; `GET /api/trash/documents` sweeps
expired rows before listing, so the visible rows were normally ineligible
for the action by construction.
Empty trash now drives `purge_all_trashed`: the caller's trashed rows
(`list_all_trashed`, no age filter) each go through the same guarded,
row-locked `purge` as the single-document delete — bytes first, then the
row, in one transaction — so a row restored mid-flight is skipped instead of
force-deleted, and an unlink failure rolls that row back and answers 500 with
a retryable message. Retention expiry stays where it was: the sweep's
`purge_candidates` is now the only age-gated selection, and the lazy
retention sweep still runs on the listing and at startup.
Tests: the router suite replaces the retention-gated expectation with the
reviewer's repro (fresh row purged, bytes unlinked, shelf and other users'
trash untouched, a failing unlink stays retryable and 500); a blocking-I/O
anchor drives the new entry point through the offload; the mocked e2e covers
the action end to end; a new real-backend spec performs it against the real
gateway and re-reads `GET /api/trash/documents`. README, API, ARCHITECTURE
and the phase-2 design docs (en+zh) state the age-independent contract.
* fix(frontend): restore sanitization in custom streamdown rehype chains
Streamdown 2.5 replaces its entire default rehype chain
[rehype-raw, rehype-sanitize, rehype-harden] with whatever array the
caller passes via the rehypePlugins prop. Every custom chain in this
repo therefore rendered LLM/stored markdown without any sanitization:
- Artifact markdown previews (markdown-preview-plugins.ts +
artifact-file-detail.tsx) parse raw HTML via rehypeRaw, so a
generated .md artifact could inject <style>/<iframe>/on* handlers
into the workspace DOM (stored XSS; only javascript: anchors were
blocked by the ArtifactLink component).
- The memory settings summary (memory-settings-page.tsx) spread the
shared preset without component overrides, so a hostile
<a href="javascript:..."> in stored memory content rendered as a
clickable anchor.
Fix strategy:
- Add rehype-sanitize (already resolved in the lockfile via streamdown)
as a direct dependency and re-insert a [rehypeSanitize, schema] step
in the shared preset (core/streamdown/plugins.ts). It runs after
rehypeRaw (raw HTML must be parsed into hast before it can be
cleaned) and before rehypeKatex/rehypeSlug (their output is trusted
and would otherwise be filtered or clobbered) - the same
raw -> sanitize -> math ordering streamdown itself uses.
- The schema extends rehype-sanitize's GitHub-style defaultSchema (the
base of streamdown's own sanitize schema) so legitimate authored
artifact HTML (tables, details, images, alignment/size attributes)
keeps working while script/iframe/style, on* handlers and
non-allow-listed URL schemes (javascript:, data:, ...) are dropped.
The only extensions are tel: hrefs and the math-inline/math-display
class markers remark-math emits and rehype-katex detects.
- Position rehypeSlug after the sanitize step in the artifact chain so
sanitize's id clobbering (id="x" -> id="user-content-x") cannot break
the heading anchors it creates.
- Pass a: createMarkdownLinkComponent() on the memory settings page as
defense in depth, matching the chat rendering path.
Unit tests feed a hostile payload (<a href="javascript:...">,
<img onerror>, <script>, <iframe>, <style>, ontoggle) through both
render paths and assert no executable/clickable equivalent survives,
plus regression guards for heading anchors, legitimate HTML and KaTeX
math rendering.
* fix(frontend): keep the sanitize clobber prefix on heading anchors; minimal lockfile
Review follow-ups on the sanitization change:
- Heading anchors: rehypeScopedSlug replaces rehype-slug in the artifact
chain. It runs after the sanitize step (so raw-HTML headings are also
anchored) but keeps rehype-sanitize's user-content- id prefix — an
untrusted heading like "## current" cannot mint an unprefixed
id="current" (the DOM-clobbering shape the sanitizer guards against).
In-page fragment links are translated to the prefixed anchors so they
still resolve; external URLs, bare "#", already-prefixed fragments and
sanitize-prefixed raw-HTML ids are left untouched.
- Lockfile: regenerated as a minimal diff — only the two direct-dependency
importer entries (rehype-sanitize, github-slugger for the scoped slug)
are added; the libc platform selectors on the 64 native package records
are preserved byte-for-byte instead of being dropped by lockfile
normalization.
Full frontend suite: 1034 tests passing; tsc and prettier clean.
* style: reorder github-slugger import ahead of the hast type import
* test(e2e): expect the clobber-prefixed heading anchor in artifact preview
The scoped slug plugin gives generated heading ids rehype-sanitize's
user-content- prefix and translates fragment links to match, so the
anchor-scroll test must locate the prefixed id.
* fix(frontend): reset the scoped slugger per tree; keep footnote anchors single-prefixed
Review follow-ups:
- The scoped slug attacher holds one GithubSlugger, but streamdown
caches the unified processor by plugin name, so the instance survived
across parses and repeated renders of the same heading grew -1/-2
suffixes (the artifact-anchor e2e could not find the id on re-render).
The transformer now resets the slugger per tree, as rehype-slug does;
a regression test renders identical artifact markdown twice.
- remark-rehype emits GFM footnote anchors already clobber-prefixed
(user-content-fn-1); the sanitize step prefixed those ids again while
their hrefs stayed single-prefixed, breaking footnote navigation in
every chain built on the shared preset. A new rehypeClobberFragments
step runs right after sanitize: double-prefixed ids are normalized
back to one prefix, and unprefixed fragment hrefs are translated to
the prefixed form (already-prefixed and external links untouched).
The artifact slug now inserts after this step; covered by a footnote
regression test on the shared render path.
Unit suite 1036 passing; artifact-preview e2e verified locally
(9/9, including the heading-anchor scroll test).
* fix(frontend): use code-aware stripLeakedSystemTags for all internal marker tags
Replace the ad-hoc /<\/?memory>/g in ClipboardSafeStreamdown with a
dedicated stripLeakedSystemTags() function in preprocess.ts that:
- Covers all INTERNAL_MARKER_TAGS (<memory>, <system-reminder>,
<current_date>, <uploaded_files>, <slash_skill_activation>) instead
of only <memory>
- Is code-aware: skips fenced code blocks (```) and indented code
blocks (4-space indent), so user-written meta-discussions about the
memory system are not silently stripped
- Handles opening tags, closing tags, self-closing tags, and tags with
attributes
- Adds 15 unit tests covering all the above cases
* fix(frontend): use marker-aware fence tracking in stripLeakedSystemTags
The boolean insideFence toggle incorrectly closes a fence on any line
matching 3+ backticks or tildes, regardless of the actual delimiter.
This causes tags inside a tilde-fenced block containing a backtick
sub-fence, or a 4-backtick block containing a 3-backtick sub-fence, to
be silently stripped.
Track the opening fence marker (character and run length) so that only
a matching marker with at least the same length closes the fence.
Adds 4 new test cases:
- Tilde fence with inner backtick fence
- 4-backtick fence with inner 3-backtick fence
- Shorter tilde closing inside longer tilde fence
- Tags stripped after real closing fence
* fix(frontend): fix TypeScript strict errors in fence marker tracking
- Use non-null assertion (!) on fenceMatch[1] (guaranteed by regex)
- Use charAt(0) instead of [0] to avoid string | undefined type
* fix(frontend): improve chat math rendering
* fix(frontend): refine math rendering stability
* fix(frontend): address review feedback on math preprocessing
- Fix escaped-backslash blind spot: consume \\\\ as a unit so \\( and
\\[ are not mis-interpreted as math delimiters when the backslash
itself is escaped.
- Thread inInlineCode state across lines so multi-line backtick code
spans are protected from delimiter conversion.
- Remove double math normalization: preprocessStreamdownMarkdown now
only handles Mermaid; math normalization lives solely in
ClipboardSafeStreamdown, preventing non-idempotent double passes.
- Wire parseIncompleteMarkdown to isLoading in MarkdownContent so
Streamdown's streaming-incomplete-math handling is actually active
during streaming.
* fix(frontend): address streamdown math review issues
* refactor(frontend): move streamdown preprocessing out of ai elements
* fix(frontend): cap deeply nested list indentation to prevent render crash
Deeply nested lists make marked's recursive list tokenizer overflow the
call stack during Streamdown's lexing useMemo, throwing an uncaught
"RangeError: Maximum call stack size exceeded" that replaces the chat
route with an error page (issue #3393); on larger stacks the same input
exhausts the heap, which the render error boundary cannot catch.
Mirror the existing capBlockquoteNesting guard with capListNesting, which
clamps leading whitespace to 200 columns (~100 nesting levels) only when
pathologically deep indentation is present, leaving normal content and
fenced code untouched. Wire both through capMarkdownNesting.
* fix(frontend): satisfy prettier format check in preprocess test
* fix(frontend): exempt indented code from list-indent cap (PR #3570 review)
* fix(frontend): keep capping all deep indentation outside fenced code
Revert the indented-code exemption from the PR #3570 review nit. Taken
literally the suggested guard (insideFence || INDENTED_CODE_RE.test(line))
no-ops capListNesting, because INDENTED_CODE_RE matches every line with
4+ leading spaces — i.e. exactly the deep-indent lines the cap targets.
A context-aware exemption (only treat 4+-space lines as code after a
blank line) instead reopens the crash: blank-separated deeply nested list
items get exempted and still blow up marked (verified: OOM at depth ~1.5k).
Unlike blockquotes (markers take <=3 leading spaces, so deep-quote lines
never look like indented code), list vs. indented-code indentation is
ambiguous line-by-line, so any exemption is exploitable. Keep capping all
deep indentation outside fenced code; the only cost is mild corruption of
a >200-column indented-code line, which never occurs in real content and
is strictly preferable to a render crash. Add a regression test locking
the blank-line case.
* fix(frontend): render user messages as plain text and cap blockquote nesting
User messages are typed or pasted plain text, not authored Markdown, but
they were rendered through the full Streamdown pipeline. Pasted source
files got fragmented (indented chunks become code blocks, paragraphs
collapse and lose indentation), "$...$" spans were KaTeX-ified, and a
message with thousands of nested ">" markers overflowed the call stack
in marked's recursive blockquote lexer, permanently crashing the thread
on every load.
Render human message content verbatim with pre-wrap instead, and cap
blockquote nesting at 100 levels at the Streamdown chokepoint so model
output cannot trigger the same recursion either.
Closes#3500
* fix(frontend): absorb marked lexer crashes with a render fallback boundary
Review found two gaps in the nesting cap: marked's list and blockquote
tokenizers are mutually recursive, so a list marker in front of the
quote chain ("- > > > ...") bypassed the blockquote-only regex and
still overflowed the stack; and the line-based rewrite was fence-blind,
silently truncating literal ">" runs inside code blocks.
Add an error boundary around Streamdown that renders the raw content as
plain pre-wrap text when rendering throws (retrying on the next content
change), keep the cap as a fast path for the dominant pure-">" case,
and make it skip fenced and indented code lines.
- 添加 rehype-raw 依赖以支持在 markdown 中渲染 HTML
Add rehype-raw dependency to support HTML rendering in markdown
- 重构 memory-settings-page，提取 formatMemorySection 函数减少重复代码
Refactor memory-settings-page by extracting formatMemorySection function to reduce code duplication
- 改进空状态显示，使用 HTML span 标签替代 markdown 斜体，提供更好的样式控制
Improve empty state display by using HTML span tags instead of markdown italics for better style control
- 为 skill-settings-page 添加完整的国际化支持，替换硬编码的英文文本
Add complete i18n support for skill-settings-page, replacing hardcoded English text
- 更新国际化文件，添加技能设置页面的空状态文本（中英文）
Update i18n files with empty state text for skill settings page (both Chinese and English)
- 在 streamdown 插件配置中添加 rehypeRaw 以支持 HTML 渲染
Add rehypeRaw to streamdown plugins configuration to support HTML rendering
Co-authored-by: Cursor <cursoragent@cursor.com>
For human messages, disable remark-gfm autolink feature to prevent
URLs from incorrectly including adjacent text (especially Chinese
characters) as part of the link. This ensures that when users input
"https://example.com 帮我分析", only the URL becomes a link.
Co-authored-by: Cursor <cursoragent@cursor.com>
