From a540ff46a93da9780d04145005bd1eda700ce801 Mon Sep 17 00:00:00 2001 From: hetaoBackend Date: Thu, 6 Aug 2026 20:24:02 +0800 Subject: [PATCH] docs: document upload generation leases --- README.md | 6 +++-- backend/AGENTS.md | 10 ++++---- backend/docs/API.md | 8 +++++- backend/docs/FILE_UPLOAD.md | 9 +++++-- backend/docs/PATH_EXAMPLES.md | 6 +++-- ...26-08-06-upload-collision-safety-design.md | 25 +++++++++++++------ ...-08-06-upload-review-remediation-design.md | 4 +-- 7 files changed, 46 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index 14df997f4..be16b6a2a 100644 --- a/README.md +++ b/README.md @@ -961,7 +961,9 @@ DeerFlow doesn't just *talk* about doing things. It has its own computer. Each task gets its own execution environment with a full filesystem view — skills, workspace, uploads, outputs. The agent reads, writes, and edits files. It can view images and, when configured safely, execute shell commands. -Uploads from the Web UI, embedded client, and IM channels share one collision-safe storage rule. A completed payload is published only if its candidate name does not exist; concurrent `report.pdf` uploads become `report.pdf`, `report_1.pdf`, `report_2.pdf`, and so on without replacing one another. Optional document conversions are system-owned assets under `/mnt/user-data/.upload-conversions/.md`. They are returned through the upload response but omitted from the primary upload listing. Deleting a primary removes only its exact generated asset and never infers that a user-uploaded sibling such as `uploads/report.md` is disposable. +Uploads from the Web UI, embedded client, and IM channels share one collision-safe storage rule. A completed payload is published only if its candidate name does not exist; concurrent `report.pdf` uploads become `report.pdf`, `report_1.pdf`, `report_2.pdf`, and so on without replacing one another. The selected name is leased through conversion and sandbox synchronization, so deleting that exact name waits for its active upload lifecycle while unrelated filenames continue concurrently. Internal staging names matching `.upload-*.part` are rejected. + +Optional document conversions are system-owned assets under `/mnt/user-data/.upload-conversions/`. Normal targets use `.md`; names that would exceed the filesystem component limit use a deterministic UTF-8-safe prefix plus the full SHA-256 digest. The exact generated path is returned through the upload response and omitted from the primary upload listing. Local and AIO sandboxes expose this namespace read-only. Deleting a primary removes only its exact generated asset and never infers that a user-uploaded sibling such as `uploads/report.md` is disposable. The built-in `grep` tool searches either one text file or all matching text files below a directory, so an agent can search an uploaded document directly without first broadening the request to the entire uploads directory. @@ -993,7 +995,7 @@ This is the difference between a chatbot with tool access and an agent with an a # Paths inside the sandbox container /mnt/user-data/ ├── uploads/ ← your primary files -├── .upload-conversions/ ← generated Markdown (hidden from upload listings) +├── .upload-conversions/ ← generated Markdown (read-only; hidden from upload listings) ├── workspace/ ← agents' working directory └── outputs/ ← final deliverables ``` diff --git a/backend/AGENTS.md b/backend/AGENTS.md index 8f9b4ffc7..9e2e2f3db 100644 --- a/backend/AGENTS.md +++ b/backend/AGENTS.md @@ -557,7 +557,7 @@ Localhost persistence deliberately reads the direct request `Host` and ignores ` | **Skills** (`/api/skills`) | `GET /` - list skills; `GET /{name}` - details; `PUT /{name}` - update enabled; `POST /install` - install from .skill archive (accepts standard optional frontmatter like `version`, `author`, `compatibility`); `POST /reload` - admin-only process-local prompt-cache invalidation after trusted external filesystem changes | | **Integrations** (`/api/integrations`) | `GET /lark/status` - inspect managed Lark/Feishu CLI integration state, including `sandbox_runtime_mode` / `sandbox_runtime_ready` (whether `lark-cli` will actually be present in the sandbox at chat time); `POST /lark/install` - admin-only install of the official `lark-*` managed skill pack; `POST /lark/config/start` and `/lark/config/complete` - internal first-time Lark connection setup; `POST /lark/auth/start` and `/lark/auth/complete` - browser device-flow user authorization without terminal access, with optional `domains` / exact `scope` for incremental permission grants | | **Memory** (`/api/memory`) | `GET /` - memory data; `POST /reload` - force reload; `GET /config` - config; `GET /status` - config + data | -| **Uploads** (`/api/threads/{id}/uploads`) | `POST /` - upload files (auto-converts PDF/PPT/Excel/Word); `GET /list` - list; `DELETE /{filename}` - delete | +| **Uploads** (`/api/threads/{id}/uploads`) | `POST /` - atomically publish files without replacing collisions and optionally convert PDF/PPT/Excel/Word; `GET /list` - list primary uploads only; `DELETE /{filename}` - wait for that filename's active publication lifecycle, then delete the primary and its exact generated conversion | | **Threads** (`/api/threads/{id}`) | `DELETE /` - remove DeerFlow-managed local thread data after LangGraph thread deletion; `POST /branches` - create a new main-thread branch from a completed assistant turn checkpoint and, when an addressable pre-user replay checkpoint exists, materialize it into the branch namespace so the inherited response remains regeneratable. Workspace files are not checkpointed, so the branch only best-effort copies the current workspace when branching from the **latest** turn (`workspace_clone_mode="current_thread_best_effort"`); branching from an older/historical turn skips the copy (`workspace_clone_mode="skipped_historical_turn"`) so the branch never inherits files that only exist in a later timeline. Thread-scoped runtime channels (`sandbox`, `thread_data`) are not copied onto the branch: the parent's `sandbox_id` binds path mappings and the release lifecycle to the parent's workspace, so the branch lazily acquires its own sandbox instead. Branch creation also seeds the new thread's run-event feed from the branch checkpoint's visible messages (`history_seed_mode` in the response): the thread feed reads run_events, not checkpoints, so without the seed the inherited history disappears from the UI after the branch's first run (#4380). Seeded rows are grouped into one synthetic run per inherited turn (`branch-seed-{thread_id}-{n}`, a new turn opening at every persisted human message, including an allowlisted hidden `ask_clarification` reply) because `run_id` is a turn identity to the feed's consumers, not a provenance tag: regenerating an inherited answer supersedes that row's whole `run_id` in `GET /messages/page`, so one shared id for the entire seed deleted the complete inherited history on a branch's first regenerate (#4458); `GET /goal`, `PUT /goal`, `DELETE /goal` - read, set, and clear the active thread goal; `POST /compact` - manually summarize older active context into `summary_text` and retain the recent message window, blocked while a run is in flight; unexpected failures are logged server-side and return a generic 500 detail | | **Artifacts** (`/api/threads/{id}/artifacts`) | `GET /{path}` - stream regular text and binary artifacts with `FileResponse`, including byte-`Range` 206/416 behavior used by bounded text previews and media seeking; active content types (`text/html`, `application/xhtml+xml`, `image/svg+xml`) are always forced as download attachments to reduce XSS risk; `?download=true` still forces download for other file types. `PUT /{path}` atomically replaces an existing UTF-8 text file under `/mnt/user-data/outputs` when its expected SHA-256 still matches; active runs conflict, and non-mounted sandbox providers receive the same update explicitly. Atomic replacement applies the existing POSIX permission handling when descriptor-based APIs are available and otherwise keeps the platform-native temporary-file permissions (Windows). | | **Suggestions** (`/api/suggestions`) | `GET /config` - returns global suggestions config boolean; `POST /threads/{id}/suggestions` - generate follow-up questions; rich list/block model content is normalized and inline reasoning (`...`, including unclosed/truncated blocks from reasoning models like MiniMax-M3) is stripped before JSON parsing | @@ -699,8 +699,8 @@ that cannot tell sibling branches apart. **Provider Pattern**: `SandboxProvider` with `acquire`, `acquire_async`, `get`, `release` lifecycle. Async agent/tool paths call async sandbox lifecycle hooks so Docker sandbox creation, discovery, cross-process locking, readiness polling, and release stay off the event loop. **Environment policy** (`sandbox/env_policy.py`): `execute_command` no longer inherits the full `os.environ`. `build_sandbox_env()` scrubs secret-looking names (`*KEY*`/`*SECRET*`/`*TOKEN*`/`*PASS*`/`*CREDENTIAL*`) from the inherited environment before layering injected request secrets on top, so platform credentials (e.g. `OPENAI_API_KEY`) never leak into skill subprocesses. Benign vars (`PATH`, `HOME`, `LANG`, `VIRTUAL_ENV`, ...) are preserved. **Implementations**: -- `LocalSandboxProvider` - Local filesystem execution. `acquire(thread_id)` returns a per-thread `LocalSandbox` (id `local:{thread_id}`) whose `path_mappings` resolve `/mnt/user-data/{workspace,uploads,outputs}` and `/mnt/acp-workspace` to that thread's host directories, so the public `Sandbox` API honours the `/mnt/user-data` contract uniformly with AIO. `acquire()` / `acquire(None)` keeps the legacy generic singleton (id `local`) for callers without a thread context. Per-thread sandboxes are held in an LRU cache (default 256 entries) guarded by a `threading.Lock`. Public, custom, legacy, and managed integration skill mappings point at stable enabled-only projection roots rather than raw skill directories. -- `AioSandboxProvider` (`packages/harness/deerflow/community/`) - Docker-based isolation. Active-cache and warm-pool entries are checked with the backend during acquire/reuse; definitively dead containers are dropped from all in-process maps so the thread can discover or create a fresh sandbox instead of reusing a stale client. Backend health-check failures are treated as unknown, not dead; local discovery likewise treats an unverifiable container as not adoptable and falls through to create rather than failing acquire. `get()` remains an in-memory lookup for event-loop-safe tool paths — it never touches the ownership store (that would be blocking IO on the event loop); ownership is published on acquire/reclaim and refreshed off the event loop by the dedicated renewal thread (`_renew_owned_leases`). `uses_thread_data_mounts` defaults to backend detection (`LocalContainerBackend=True`, remote/provisioner backends=False), while the optional `sandbox.thread_data_mounts` boolean takes precedence for deployments that guarantee the Gateway and sandbox share the same thread user-data directories. Setting it `true` skips upload-time sandbox acquire/sync; a false positive leaves uploads unavailable to the sandbox. Local-container and hostPath-provisioner mounts use the same stable skill projection roots; PVC-backed skills remain governed by the operator-supplied PVC layout until PVC materialization is implemented. Readiness probes and `agent_sandbox` clients classify loopback/private IPs, single-label cluster hosts, and Docker/Podman internal hostnames as direct control-plane destinations and set `trust_env=False`; external FQDNs and public IPs retain environment proxy support. +- `LocalSandboxProvider` - Local filesystem execution. `acquire(thread_id)` returns a per-thread `LocalSandbox` (id `local:{thread_id}`) whose `path_mappings` resolve `/mnt/user-data/{workspace,uploads,outputs}` and `/mnt/acp-workspace` to that thread's host directories, so the public `Sandbox` API honours the `/mnt/user-data` contract uniformly with AIO. The more-specific `/mnt/user-data/.upload-conversions` mapping is read-only even though the aggregate `/mnt/user-data` mapping is writable. `acquire()` / `acquire(None)` keeps the legacy generic singleton (id `local`) for callers without a thread context. Per-thread sandboxes are held in an LRU cache (default 256 entries) guarded by a `threading.Lock`. Public, custom, legacy, and managed integration skill mappings point at stable enabled-only projection roots rather than raw skill directories. +- `AioSandboxProvider` (`packages/harness/deerflow/community/`) - Docker-based isolation. Active-cache and warm-pool entries are checked with the backend during acquire/reuse; definitively dead containers are dropped from all in-process maps so the thread can discover or create a fresh sandbox instead of reusing a stale client. Backend health-check failures are treated as unknown, not dead; local discovery likewise treats an unverifiable container as not adoptable and falls through to create rather than failing acquire. `get()` remains an in-memory lookup for event-loop-safe tool paths — it never touches the ownership store (that would be blocking IO on the event loop); ownership is published on acquire/reclaim and refreshed off the event loop by the dedicated renewal thread (`_renew_owned_leases`). `uses_thread_data_mounts` defaults to backend detection (`LocalContainerBackend=True`, remote/provisioner backends=False), while the optional `sandbox.thread_data_mounts` boolean takes precedence for deployments that guarantee the Gateway and sandbox share the same thread user-data directories. Setting it `true` skips upload-time sandbox acquire/sync; a false positive leaves uploads unavailable to the sandbox. Thread mounts create and expose `.upload-conversions` explicitly as read-only, separate from the writable uploads mount. Local-container and hostPath-provisioner mounts use the same stable skill projection roots; PVC-backed skills remain governed by the operator-supplied PVC layout until PVC materialization is implemented. Readiness probes and `agent_sandbox` clients classify loopback/private IPs, single-label cluster hosts, and Docker/Podman internal hostnames as direct control-plane destinations and set `trust_env=False`; external FQDNs and public IPs retain environment proxy support. - `E2BSandboxProvider` (`packages/harness/deerflow/community/e2b_sandbox/`) provides E2B remote isolation. New sandboxes receive a one-shot upload from the enabled-only public, custom, legacy, and managed integration projections. Existing E2B VMs keep their @@ -765,9 +765,9 @@ that cannot tell sibling branches apart. **Shared warm-pool lifecycle:** community sandbox providers that keep released sandboxes alive for fast reuse share `deerflow.community.warm_pool_lifecycle.WarmPoolLifecycleMixin`. The mixin owns the common `DEFAULT_IDLE_TIMEOUT=600`, `IDLE_CHECK_INTERVAL=60`, `DEFAULT_REPLICAS=3`, idle-checker loop, warm-pool expiry, oldest-warm eviction, replica counting, and soft-cap logging. Providers remain responsible for their own active registries, creation/discovery, health checks, and destroy hook (`_destroy_warm_entry`): AIO destroys `SandboxInfo` through its backend; Boxlite closes loop-affine `BoxliteBox` handles; Tenki closes the microVM session (`TenkiSandbox.close`, which terminates the remote sandbox). AIO keeps active-idle cleanup outside the mixin and delegates only warm-pool expiry to the shared helper. **Virtual Path System**: -- Agent sees: `/mnt/user-data/{workspace,uploads,outputs}`, `/mnt/skills` +- Agent sees: `/mnt/user-data/{workspace,uploads,outputs}`, read-only `/mnt/user-data/.upload-conversions`, and `/mnt/skills` - Physical: `backend/.deer-flow/users/{user_id}/threads/{thread_id}/user-data/...`; raw skills stay under `deer-flow/skills/` and managed integration storage, while sandboxes read `backend/.deer-flow/skills_view/public/` and `backend/.deer-flow/users/{user_id}/skills_view/{custom,legacy,integrations}/` -- Translation: `LocalSandboxProvider` builds per-thread `PathMapping`s for the user-data prefixes at acquire time; `tools.py` keeps `replace_virtual_path()` / `replace_virtual_paths_in_command()` as a defense-in-depth layer (and for path validation). AIO has the directories volume-mounted at the same virtual paths inside its container, so both implementations accept `/mnt/user-data/...` natively. +- Translation: `LocalSandboxProvider` builds per-thread `PathMapping`s for the user-data prefixes at acquire time; `tools.py` keeps `replace_virtual_path()` / `replace_virtual_paths_in_command()` as a defense-in-depth layer (and for path validation). AIO has the directories volume-mounted at the same virtual paths inside its container, so both implementations accept `/mnt/user-data/...` natively. Both providers mount `.upload-conversions` read-only; only host upload-conversion code may mutate generated files or the stable `.locks/.lock` coordination files. - Detection: `is_local_sandbox()` accepts both `sandbox_id == "local"` (legacy / no-thread) and `sandbox_id.startswith("local:")` (per-thread) **Sandbox Tools** (in `packages/harness/deerflow/sandbox/tools.py`): diff --git a/backend/docs/API.md b/backend/docs/API.md index d02b65fd6..000486173 100644 --- a/backend/docs/API.md +++ b/backend/docs/API.md @@ -636,7 +636,9 @@ Content-Type: multipart/form-data - Excel (`.xls`, `.xlsx`) - Word (`.doc`, `.docx`) -All upload entry points publish complete payloads without replacing an existing name. Concurrent collisions are returned as `document.pdf`, `document_1.pdf`, `document_2.pdf`, and so on. Generated Markdown is stored outside the primary namespace and is not returned by the list endpoint. Deleting `document.pdf` also deletes only `.upload-conversions/document.pdf.md`; an independent `uploads/document.md` is preserved. +All upload entry points publish complete payloads without replacing an existing name. Concurrent collisions are returned as `document.pdf`, `document_1.pdf`, `document_2.pdf`, and so on. A published filename remains leased through conversion, permission adjustment, sandbox synchronization, and response construction; deletion of that exact filename waits for the active lifecycle, while other filenames remain independent. Basenames matching the internal `.upload-*.part` staging pattern are rejected. + +Generated Markdown is stored outside the primary namespace and is not returned by the list endpoint. Normal conversion names are `.md`; if that component would exceed 255 UTF-8 bytes, the response contains a deterministic UTF-8-safe prefix plus the full SHA-256 digest and `.md`. Clients must consume the returned `markdown_*` fields rather than derive the path. Local and AIO sandboxes mount `.upload-conversions` read-only. Deleting `document.pdf` also deletes only its exact generated conversion; an independent `uploads/document.md` is preserved. #### List Uploaded Files @@ -676,6 +678,10 @@ DELETE /api/threads/{thread_id}/uploads/{filename} } ``` +If an upload, conversion, or sandbox synchronization still owns this exact filename, the +delete waits for that lifecycle to finish before removing the primary and its generated +conversion. Work on unrelated filenames is not serialized. + ### Thread Cleanup Remove DeerFlow-managed local thread files under `.deer-flow/threads/{thread_id}` after the LangGraph thread itself has been deleted. diff --git a/backend/docs/FILE_UPLOAD.md b/backend/docs/FILE_UPLOAD.md index 1e62b80c7..777986cb7 100644 --- a/backend/docs/FILE_UPLOAD.md +++ b/backend/docs/FILE_UPLOAD.md @@ -52,7 +52,9 @@ POST /api/threads/{thread_id}/uploads - `virtual_path`: Agent 在沙箱中使用的虚拟路径 - `artifact_url`: 前端通过 HTTP 访问文件的 URL -所有上传入口都先完整写入同目录暂存文件，再以“不替换已有条目”的原子操作发布。同名碰撞依次命名为 `document.pdf`、`document_1.pdf`、`document_2.pdf`；响应中的 `filename` 和各路径字段始终使用实际发布名。 +所有上传入口都先完整写入同目录暂存文件，再以“不替换已有条目”的原子操作发布。同名碰撞依次命名为 `document.pdf`、`document_1.pdf`、`document_2.pdf`；响应中的 `filename` 和各路径字段始终使用实际发布名。系统内部保留 `.upload-*.part` 作为暂存命名空间，用户上传使用该模式的 basename 会在创建暂存文件前被拒绝。 + +实际发布名会在转换、权限调整、沙箱同步和响应构造期间持有同名租约。删除该名称会等待当前生命周期完成；其他文件名仍可并发处理。跨进程协调使用 `.upload-conversions/.locks/` 下稳定保留的摘要锁文件，该目录属于内部实现，不应由 Agent 或部署脚本修改或清理。 ### 2. 查询上传限制 ``` @@ -108,7 +110,7 @@ DELETE /api/threads/{thread_id}/uploads/{filename} } ``` -删除 `document.pdf` 时，只会额外删除它精确拥有的 `.upload-conversions/document.pdf.md`。系统不会推断或删除 `uploads/document.md`；该文件可能是用户独立上传的内容。 +删除 `document.pdf` 时，会先等待该实际文件名当前正在进行的上传、转换或沙箱同步生命周期结束，然后只额外删除它精确拥有的生成资产。系统不会推断或删除 `uploads/document.md`；该文件可能是用户独立上传的内容。其他文件名不会被这次等待阻塞。 ## 支持的文档格式 @@ -128,6 +130,8 @@ Deletion: 删除 report.pdf 时只删除 .upload-conversions/report.pdf.md； /mnt/user-data/uploads/report.md 永远不会被推断为生成文件或自动删除。 ``` +通常生成名为 `<实际主文件名>.md`。如果这一文件名组件会超过 255 个 UTF-8 字节，系统会使用 UTF-8 安全截断的主文件名前缀、完整 SHA-256 摘要和 `.md`，并在响应中返回精确的 `markdown_*` 路径。客户端和 Agent 不应自行拼接生成路径。Local 与 AIO 沙箱都将 `.upload-conversions` 显式挂载为只读；只有 DeerFlow 宿主进程中的转换代码可以写入生成文件和内部锁。 + 默认情况下，自动转换是关闭的，以避免在网关主机上对不受信任的 Office/PDF 上传执行解析。只有在受信任部署中明确接受此风险时，才应将 `uploads.auto_convert_documents` 设置为 `true`。 ## Agent 集成 @@ -177,6 +181,7 @@ read_file(path="/mnt/user-data/.upload-conversions/document.pdf.md") 上传流程采用“线程目录优先”策略： - 先写入 `backend/.deer-flow/threads/{thread_id}/user-data/uploads/` 作为权威存储 - 本地沙箱（`sandbox_id=local`）直接使用线程目录内容 +- Local 与 AIO 的挂载模式会把 `/mnt/user-data/.upload-conversions` 单独映射为只读，即使 `/mnt/user-data` 或主上传目录可写 - 默认情况下，非本地沙箱通过 `acquire_async` 获取后，再额外同步到 `/mnt/user-data/uploads/*`，确保运行时可见 - 如果 Gateway 与远端沙箱保证挂载同一份线程 user-data（例如正确对齐的共享 PVC、NFS 或 hostPath），可设置 `sandbox.thread_data_mounts: true`；上传路由会跳过 sandbox acquire 和逐文件同步 - 不确定挂载关系时应省略该配置并保留自动检测。错误地设为 `true` 会导致文件只存在于 Gateway 存储、沙箱内不可见 diff --git a/backend/docs/PATH_EXAMPLES.md b/backend/docs/PATH_EXAMPLES.md index a019c22a3..012faffb1 100644 --- a/backend/docs/PATH_EXAMPLES.md +++ b/backend/docs/PATH_EXAMPLES.md @@ -281,8 +281,10 @@ function FileUploadList({ threadId }: { threadId: string }) { 4. **Markdown 转换** - 转换成功时，会返回额外的 `markdown_*` 字段 - - 生成文件位于 `.upload-conversions/<完整主文件名>.md`，不会出现在主文件列表中 + - 常规生成文件位于 `.upload-conversions/<完整主文件名>.md`；超长名称使用 UTF-8 安全前缀和完整 SHA-256 摘要，因此始终以上传响应中的 `markdown_*` 字段为准 + - `.upload-conversions` 在 Local 与 AIO 沙箱内只读，并且不会出现在主文件列表中 - 同名主文件按 `file.pdf`、`file_1.pdf`、`file_2.pdf` 原子发布，不会覆盖 - - 删除主文件只删除其精确生成资产，不会删除用户上传的 `uploads/file.md` + - 删除主文件会等待该实际文件名的活跃生命周期，然后只删除其精确生成资产，不会删除用户上传的 `uploads/file.md` + - `.upload-*.part` 是内部暂存名称，不能作为用户上传 basename - 建议优先使用 Markdown 版本（更易处理） - 原始文件始终保留 diff --git a/docs/superpowers/specs/2026-08-06-upload-collision-safety-design.md b/docs/superpowers/specs/2026-08-06-upload-collision-safety-design.md index 2e89a5a57..e24e69398 100644 --- a/docs/superpowers/specs/2026-08-06-upload-collision-safety-design.md +++ b/docs/superpowers/specs/2026-08-06-upload-collision-safety-design.md @@ -1,7 +1,7 @@ # Upload Collision Safety Design **Issue:** #3750 -**Status:** Approved +**Status:** Implemented; awaiting independent review **Scope:** Gateway uploads, embedded client uploads, inbound IM attachments, generated Markdown conversions, outline lookup, and upload deletion ## Problem @@ -52,6 +52,10 @@ Generated Markdown is stored outside the primary upload namespace: /user-data/.upload-conversions/.md ``` +If the normal generated component would exceed 255 UTF-8 bytes, the filename is +`..md`. Every response, outline, deletion, and +virtual-path caller uses the shared layout helper and therefore receives the same target. + Examples: ```text @@ -73,8 +77,8 @@ Candidate suffixes are inserted before the final suffix, matching the current na behavior: `archive.tar.gz` becomes `archive.tar_1.gz`, a name without an extension becomes `name_1`, and `.env` becomes `.env_1`. -A small upload-layout module will own construction of physical paths, virtual paths, and -artifact URLs. Callers will not assemble conversion paths or guess sibling names. +A small upload-layout module owns construction of physical paths, virtual paths, and +artifact URLs. Callers do not assemble conversion paths or guess sibling names. ## Atomic Publication @@ -101,7 +105,10 @@ naming mechanism. The publisher returns the actual filename chosen. Gateway, embedded client, generic IM, Feishu, DingTalk, and WeChat download staging all use it instead of implementing their -own scan-then-write flow. +own scan-then-write flow. Names matching `.upload-*.part` are rejected before staging. +Lifecycle-aware callers retain an exclusive per-name lease through later conversion, +permission, sandbox synchronization, and response work. Stable digest-named lock files +under `.upload-conversions/.locks/` coordinate both threads and processes. ## Conversion Publication and Ownership @@ -125,7 +132,8 @@ not fall back to `.md`, because that would reintroduce ambiguous ownership ## Deletion -Deleting an upload performs these exact operations: +Deleting an upload acquires the same actual-name lease, waiting only for work on that +filename, and then performs these exact operations: 1. Validate and remove `uploads/` without following symlinks. 2. Derive and remove `.upload-conversions/.md`. @@ -168,8 +176,9 @@ parallel messages cannot overwrite one another before the thread upload copy occ ### Sandbox synchronization The existing primary upload sync remains unchanged. Generated conversion sync uses its -exact virtual path under `/mnt/user-data/.upload-conversions/`. Mounted providers need no -copy; non-mounted providers sync the exact file explicitly. +exact virtual path under `/mnt/user-data/.upload-conversions/`. Local and AIO providers +mount that namespace explicitly read-only; non-mounted providers sync the exact file +explicitly. ## Error Handling @@ -185,7 +194,7 @@ copy; non-mounted providers sync the exact file explicitly. - Existing primary upload URLs and virtual paths remain unchanged. - New conversions receive a different virtual path. Responses return the exact path; clients must consume it instead of deriving a sibling name. Documentation and tests - will make this contract explicit. + make this contract explicit. - Upload listings continue to show only primary uploads. - Legacy sibling conversions remain readable as ordinary uploads but are not treated as generated assets and are never automatically deleted. diff --git a/docs/superpowers/specs/2026-08-06-upload-review-remediation-design.md b/docs/superpowers/specs/2026-08-06-upload-review-remediation-design.md index bc07518ff..c0bcffba6 100644 --- a/docs/superpowers/specs/2026-08-06-upload-review-remediation-design.md +++ b/docs/superpowers/specs/2026-08-06-upload-review-remediation-design.md @@ -2,7 +2,7 @@ **Issue:** #3750 **PR:** #4704 -**Status:** Written specification approved; implementation in progress +**Status:** Implemented; awaiting independent review **Scope:** Findings from the independent review of the collision-safe upload implementation ## Problem @@ -139,7 +139,7 @@ Normal conversion targets remain `.md`. If that compone exceed 255 UTF-8 bytes, the layout helper uses: ```text -..md +..md ``` The digest makes the shortened mapping deterministic and collision-resistant; all response,