* feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash
Implements docs/superpowers/specs/2026-09-12-projects-mvp-phase2-design.md
(issue #5160, tracker #5129) in the slice order of the spec's §16.
Slices:
- A: ProjectsConfig + write-time 422 UTF-8 byte cap; PROJECT_CONTEXT_KEY
admission pinning (both server-owned sets + worker hoist); latest-only
request-scoped <project> block via DynamicContextMiddleware
wrap_model_call/awrap_model_call (idempotent reassembly, reserved ID
prefix + marker + provenance, never persisted); journal audit
fingerprints; Instructions tab.
- B: ProjectDocumentRow + migration 0023; ProjectDocumentRepository with
locked check-and-set; hash-qualified immutable shelf storage with
Paths helpers; upload/list/content/delete-to-trash routes; project
delete trashes the shelf in-transaction; request-scoped bounded
<documents> index with honest count/shown + actionable overflow note;
list_project_documents/read_project_document tools registered only on
pinned runs; PAT allowlist + drift guards; blocking-IO anchors.
- C: shared thread-upload ingestion service (uploads router refactored to
parity); POST from-thread with provenance; attach-to-thread with
lock-staged copy (archived source allowed); read-only thread-files
view with per-group truncation reporting.
- D: restore (restored/merged/not_found/no_target/content_missing; no
file moves), purge (continuous row lock across unlink/delete/commit,
retryable on FS errors), retention sweep (lazy + startup, 24h orphan
guard, row-side reconciliation never deletes).
- E: Documents tab (shelf + conversation-files browser, provenance,
archived banner, content-missing rows), /workspace/trash route,
sidebar entry, composer attach handoff, i18n (en-US/zh-CN), e2e mocks
+ specs.
Review hardening folded in (10 rounds, all with tests):
- force active shelf content (HTML/XML family) to download; nosniff on
artifact + content responses; unified unsandboxed-iframe PDF preview
(fixes the pre-existing Chromium sandbox blank in the artifact viewer)
- scope document trash to the URL project under the document lock
- atomic no-overwrite filename reservation for ALL ingestion (seeded
claims + os.link commit with suffix retry; same-name re-upload now
unique-names instead of replacing); hidden staging only, no visible
placeholders; lease cleanup on setup failure
- serialize conversion under the document lock with post-lock active
revalidation; drain locked filesystem work on cancellation; preserve
bytes when an insert's commit state is uncertain (including trashed
rows)
- original-integrity checks before serving text or cached conversions;
content_missing surfaced in list responses (UI reads the flag, no
409-probe); downloads always serve original bytes
- bounded streaming document reads with cached char counts; shelf limits
declared in middleware release identity
- thread-root confinement for from-thread sources; config fallback
rejects fractional/infinite values; composer counts staged
attachments; pending attachments persist until submission or removal;
in-flight instruction/rename edits survive save refetches; shelf and
trash pagination; conversation-file and thread-files pages stay
subscribed to refetches
Docs: README/README_zh, backend API.md/ARCHITECTURE.md, AGENTS.md
contracts, config.example.yaml projects block.
Review follow-ups (head b4807477 → this revision):
- The trash retention sweep is split so repeated lazy triggers stay
bounded: the indexed expiry purge still runs on every trigger
(GET /api/trash/documents, POST /api/trash/purge) while the
O(all rows + all files) reconciliation is throttled to one run per
user per 15 minutes (process-local, per-user window). The startup
sweep now runs as a background task instead of blocking gateway
readiness, and shutdown awaits it (bounded).
- The export scrub (stripInternalMarkers) is fence- and indentation-aware
like the render path, so a pasted, fenced <project>/<documents> snippet
survives markdown export while real injected blocks (never fenced) are
still removed. Fence regexes moved to a dependency-free leaf module to
avoid the messages↔streamdown import cycle.
- The artifact viewer's PDF iframe no longer carries an added title
attribute (the upstream e2e contract locates it via :not([title])), and
the upstream artifact-preview spec now pins the new contract: PDFs
render unsandboxed, images keep sandbox="".
* fix(projects): round-2 review — cancel an overrun trash sweep, restore the PDF frame title
- Shutdown cancelled only the shield around the background startup sweep,
so an all-users reconciliation that outlived the 5s budget kept walking
rows and files while the document repo and DB engine were disposed
underneath it. The wait now lives in `_shutdown_startup_trash_sweep`,
which cancels the task and drains it before worker exit: the shield
keeps the wait bounded, the cancel makes it final (CancelledError lands
at the sweep's next await, and `_run_startup_trash_sweep` only catches
`Exception`, so nothing swallows it).
- The browser-preview iframe lost `title={getFileName(filepath)}` in the
previous fix round, leaving the PDF frame without an accessible name
while its siblings keep theirs. Restore it (WCAG frame titles), assert
it in the DOM test, and anchor the e2e on `iframe[title="report.pdf"]`
instead of `iframe:not([title])`.
* fix(projects): round-3 review — report the sweep's late finish, not a phantom cancel
`Task.cancel()` returns False when the sweep already finished inside the
window between the deadline firing and the cancel, so the shutdown log
claimed a cancellation that never happened. Branch on that outcome: the
warning stays for a real cancel, a late finish is logged at info, and both
paths still reap the task before worker exit.
* fix(projects): round-4 review — make Empty trash delete what it confirms
`POST /api/trash/purge` only ran the retention sweep, and the sweep's
candidate selection is age-gated, so a freshly trashed document survived
"Empty trash" even though the confirmation promises that every listed
document is permanently deleted. With one trashed row the route answered
`{"purged": 0}` and left it in place; `GET /api/trash/documents` sweeps
expired rows before listing, so the visible rows were normally ineligible
for the action by construction.
Empty trash now drives `purge_all_trashed`: the caller's trashed rows
(`list_all_trashed`, no age filter) each go through the same guarded,
row-locked `purge` as the single-document delete — bytes first, then the
row, in one transaction — so a row restored mid-flight is skipped instead of
force-deleted, and an unlink failure rolls that row back and answers 500 with
a retryable message. Retention expiry stays where it was: the sweep's
`purge_candidates` is now the only age-gated selection, and the lazy
retention sweep still runs on the listing and at startup.
Tests: the router suite replaces the retention-gated expectation with the
reviewer's repro (fresh row purged, bytes unlinked, shelf and other users'
trash untouched, a failing unlink stays retryable and 500); a blocking-I/O
anchor drives the new entry point through the offload; the mocked e2e covers
the action end to end; a new real-backend spec performs it against the real
gateway and re-reads `GET /api/trash/documents`. README, API, ARCHITECTURE
and the phase-2 design docs (en+zh) state the age-independent contract.
* fix(messages): drop legacy <uploaded_files> tag handling (#4212)
PR #4174 unified upload-context injection on <current_uploads> (IM and web
both flow through UploadsMiddleware), and #4632 documented the current
path. This removes the remaining backward-compat parsing of the
pre-#4174 <uploaded_files> tag, the final cleanup item tracked by the
issue:
- deermem: only <current_uploads> is stripped from human turns before
memory persistence, and the upload-sentence scrubber drops the legacy
tag alternative.
- mem0: the mirrored message filter recognises only <current_uploads>.
- InputSanitizationMiddleware: remove the legacy tag from the blocked-tag
denylist (it existed only because deermem parsed the old tag).
- frontend: stripUploadedFilesTag / stripInternalMarkers /
parseUploadedFiles and the message-list fallback parse only
<current_uploads>; demo thread fixtures are migrated to the current tag.
Scope decision: a <uploaded_files> block in pre-#4174 history is now
treated as ordinary user content (pinned by tests in both layers) instead
of being silently dropped or stripped.
* style: apply prettier formatting to stripUploadedFilesTag
* fix(uploads): keep legacy <uploaded_files> stripping for display/export only
Addresses review feedback on #4826: removing the legacy tag from the
frontend display layer made pre-#4174 threads render raw <uploaded_files>
XML (with server-side upload paths) in chat, copy data, and JSON exports.
The backend cleanup stands — memory pipelines and the sanitization denylist
treat only <current_uploads> as an internal marker. The frontend keeps the
legacy spelling in its display/export-only utilities
(stripUploadedFilesTag / INTERNAL_MARKER_TAGS / parseUploadedFiles and the
message-list fallback) so old history renders cleanly without leaking
internal paths, while the memory/sanitization scope-decision tests remain
unchanged.
Frontend tests now pin both spellings: <current_uploads> and legacy
<uploaded_files> are stripped from copy data, markdown leak-stripping, and
JSON exports.
* docs(ui): record accepted display-spoof tradeoff for legacy upload tag
Review note (willem-bd): since <uploaded_files> is off the sanitization
denylist, a live user can type the legacy spelling and fabricate file
chips / hide their own message text in display. Display-only and
self-inflicted with no backend semantics, so it is accepted for now;
documented at both the message-list fallback and stripUploadedFilesTag.
Age-gating the legacy spelling remains a possible follow-up.
---------
Co-authored-by: betterkite <313258397+betterkite@users.noreply.github.com>
* perf(frontend): cache settled copy-data derivation across streaming chunks
Every SSE values chunk re-renders MessageList, and the re-render re-derived
copy/toolbar text for every settled row: getAssistantTurnCopyData re-ran the
O(turn bytes) content extraction per settled group, and MessageListItem's
toolbar recomputed getMessageCopyData per message. Settled group arrays keep
their identity across chunks (deriveStableMessageGroups), so both derivations
now cache on that stable reference: a WeakMap keyed on the messages array for
turn copy data, and a useMemo on message identity for the toolbar copy text.
Fixes#5094
* fix(frontend): gate row copy-data memo and correct cache win claim
Address review: derive one memoized copy value only when
isHuman || (!isLoading && showCopyButton) and reuse it for both editing
and the toolbar, so settled assistant rows (whose toolbar never renders)
skip the derivation and human rows derive once, not twice; correct the
assistantTurnCopyDataCache comment — the regex/trim split is already
cached per message, the cache's win is the traversal/allocations for
string turns and the uncached O(bytes) map/join/trim for array-content
turns (benchmarked: 5.1x / 17.2x per settled history sweep).
* style(frontend): expand single-line messages array for Prettier
Two related fixes in the reasoning extraction path of
core/messages/utils.ts:
1. splitInlineReasoning's first pass stripped every closed
<think>...</think> pair unconditionally, so a message that discusses
the tag literally in markdown inline code (e.g. "Wrap your reasoning
in `<think>...</think>`") had its code span hollowed out and the
inner text shipped to the Reasoning panel. The streaming pass already
guards backtick-adjacent openers; apply the same guard to the
closed-pair pass so both passes agree on what counts as literal tag
talk.
2. getAssistantTurnCopyData fell back to reasoning via
`content ?? reasoning`, but extractContentFromMessage never returns
null, so the fallback was dead code and a reasoning-only turn (e.g.
stopped mid-thinking) rendered no copy button at all - inconsistent
with getMessageCopyData, which does copy reasoning in that case. Use
the same empty-string check it uses.
parseUploadedFiles stopped the filename capture at the first "("
([^\n(]+), so an entry like "- photo (1).png (12.3 KB)" failed to
match and the file silently disappeared from the message's file chips.
Browsers produce such names for duplicate downloads, making this a
common real-world shape.
Anchor the size group on the "(<number> <unit>)" pair the backend
emits (uploads_middleware formats sizes as "%.1f KB"/"%.1f MB") and
let the filename match greedily up to it, so parenthesized filenames
parse correctly.
* fix(frontend): strip and parse the <current_uploads> upload context tag
#4174 renamed the upload context block that UploadsMiddleware prepends to
the user message from <uploaded_files> to <current_uploads>, but the
frontend tag vocabulary was not updated, so the raw block (file list plus
usage guidance) rendered inside the user's chat bubble and the file-chip
content fallback stopped matching. Add the new tag to
stripUploadedFilesTag, INTERNAL_MARKER_TAGS (covers export and streamdown
preprocessing), parseUploadedFiles, and the chip fallback detection.
<uploaded_files> stays supported for history and IM-channel messages.
* fix(frontend): parse upload-context sizes to bytes for the file chip
parseUploadedFiles stored the raw leading number of the human-readable
size (e.g. parseInt("177.6 KB") -> 177) into FileInMessage.size, which
is documented as bytes. On the content-fallback chip path (history/IM
messages without additional_kwargs.files) formatBytes then rendered a
177.6 KB file as "0.2 KB".
Convert the backend's "<n> KB" / "<n> MB" form back to bytes so the chip
re-renders at the original magnitude; update the parse tests to assert
byte values.
* fix(frontend): keep leading orphan tool messages visible
#3880 stopped dropping orphan tool messages that arrive after a terminal
group, but left the leading-orphan branch dropping the message with a
console.error on every render. The case is reachable: history pagination
cuts by event seq, not turn boundaries, so the first loaded page can
begin mid-turn with tool results whose AI tool-call message sits on an
unloaded older page — in dev mode the diagnostic surfaces as a
full-screen Next.js error overlay (#4399). Open a processing group for
the leading orphan instead; loading the older page re-groups it under
its real turn.
* test(frontend): lock leading-orphan accumulation invariant; clarify self-heal comment
Address review on #4408:
- The self-heal note only holds for the pagination path. The
hidden-only-precedent case and a truly orphaned tool have no older page to
load, so the processing group persists and renders as an empty
ChainOfThought shell (convertToSteps emits steps only for type === ai).
Document that as an accepted degradation instead of implying it always
self-heals.
- Add a test locking the accumulation invariant: a leading orphan followed by
a real tool-call turn folds into one processing group, not a stranded empty
group beside the real turn.
A reasoning+content AI message with no tool calls was grouped into both an
`assistant:processing` group (the ChainOfThought panel above the bubble) and an
`assistant` group (the bubble's own Reasoning collapsible), so its reasoning
rendered twice. Such a message now becomes only the assistant bubble; the
ChainOfThought panel keeps handling intermediate reasoning and tool steps.
Closes#3868
* fix(frontend): keep orphan tool messages visible
LangGraph `messages-tuple` stream mode can emit tool-result events
out of order or replay them from subagent state (e.g. the bash subagent
under LocalSandboxProvider with allow_host_bash: true). When that
happens, the tool message arrives after a terminal assistant/human
group, so getMessageGroups' lastOpenGroup() returns null.
The previous behaviour was console.error + drop, which silently hid
the tool result from the UI - the user could not see the tool output
or tool-call records even though the agent executed the action
correctly (backend + Langfuse trace were both fine).
Fallback now attaches the orphan tool message to the most recent group
so the UI shows it. Adds a unit test that covers the orphan-tool path
and a duplicate-stream regression test.
* test(frontend): make orphan-tool replay test actually reach the fallback
The previous fixture was `human → ai(tool_calls) → tool → tool(replay)`
with no terminal group in between, so both tool messages hit the
unchanged happy path (`open.messages.push(...)`). Neither message was
an orphan, the new `else if (groups.length > 0)` fallback was never
exercised, and the `>= 1` length assertion was satisfied trivially by
t-1a alone.
Interleave a terminal assistant message between the original tool
result and the replayed one, so t-1b arrives when lastOpenGroup()
returns null. Now the strict assertion that t-1b is reachable can only
pass via the new fallback branch.
Review feedback from @willem-bd on PR #3880.
* fix(frontend): satisfy noUncheckedIndexedAccess in orphan-tool fallback
The fallback branch pushed into `groups[groups.length - 1].messages`
directly, which trips `noUncheckedIndexedAccess` under the project's
strict TS config and breaks lint-frontend, e2e-tests, and the
full-stack render CI layer.
Take `groups[groups.length - 1]` into a local `lastGroup` and check
for `undefined` explicitly. The `else if (groups.length > 0)` guard
becomes redundant once `lastGroup` is checked, so the branches are
folded into the outer `else` to keep the structure flat. Behavior is
unchanged: orphan tools still attach to the most recent group, and
the empty-groups diagnostic still logs at ERROR level.
Review feedback from @willem-bd on PR #3880.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(frontend): render full content for multi-part AI messages
Gemini streams the first content block as a {type:text} object carrying
the thinking signature, then emits continuation deltas as bare strings.
The content extractors only kept {type:text} objects and dropped the
string parts, truncating each AI message to its first block.
Handle string elements in extractContentFromMessage, extractTextFromMessage,
and textOfMessage so the full message renders.
Fixes#1000
* test(frontend): cover multi-part AI message content extraction
Add a regression test for the #1000 truncation: an AI message whose
content is [{type:text, signature}, "bare string"] (Gemini's shape after
LangChain merge_content). Fails on main, passes with the extractor fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(frontend): cover textOfMessage multi-part content extraction
Add tests for the bare-string continuation case and the null-when-empty
contract, and document why textOfMessage joins flat ("") while the body
extractors join with "\n". Addresses review feedback on #3649.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(frontend): strip unclosed <think> tags from streaming AI content
During streaming, an opening <think> tag may arrive in one chunk
while the matching </think> arrives in a later chunk. The existing
splitInlineReasoning regex only matched fully closed pairs, so the
mid-flight reasoning was left in message.content and rendered into
the chat bubble via the markdown pipeline's rehypeRaw plugin until
the closing tag landed.
Extend splitInlineReasoning with a second pass: after stripping every
closed <think>...</think> pair, route any remaining content from a
lone opener to the reasoning slot and leave only the preceding
preamble in content. Closed-tag behavior is unchanged.
Covers every provider whose stream emits reasoning inline as <think>
tags (MiniMax streaming path, MindIE, PatchedChatOpenAI, and any
gateway-served DeepSeek/OpenAI-compatible model).
* style(frontend): apply prettier formatting to streaming reasoning tests
* fix(frontend): skip <think> split for literal think tags in inline code
Treats a `<think>` opener immediately preceded by a backtick as part of
markdown inline code rather than a streaming reasoning marker. Prevents
permanent content truncation when an AI message documents the `<think>`
tag literally (e.g. ``Use `<think>` markers``), where the streaming-safe
fallback would otherwise route the rest of the answer into the reasoning
panel because no `</think>` ever arrives.
Adds regression tests for both the post-stream and mid-stream cases.
* fix(middleware): Prevent todo completion reminder IMMessage leak (#2892)
* make format
* fix(middleware): Clear stale todo reminder counts (#2892)
* add size guard for _completion_reminder_counts and add a integration test
