29 Commits

Author SHA1 Message Date
hetaoBackend
f0e7e4fcda fix: preserve raced hardlink upload targets 2026-08-07 06:31:39 +08:00
hetaoBackend
4f5dcd61dc fix: harden upload deletion recovery retries 2026-08-07 06:16:10 +08:00
hetaoBackend
e579c95356 fix: repersist upload deletion phase markers 2026-08-07 05:57:26 +08:00
hetaoBackend
5d4d75ad9b fix: persist upload deletion recovery roots 2026-08-07 05:47:31 +08:00
hetaoBackend
3898cc0a5c fix: make upload deletion recovery crash-consistent 2026-08-07 05:36:31 +08:00
hetaoBackend
de9f9eb941 fix: persist remote deletion finalization guards 2026-08-07 05:15:49 +08:00
hetaoBackend
5650a80b0e fix: bind remote deletion recovery to sandbox incarnations 2026-08-07 04:57:40 +08:00
hetaoBackend
f6ec5d4ce6 fix: make remote deletion recovery provider safe 2026-08-07 04:13:55 +08:00
hetaoBackend
732cd95f83 fix: bind remote upload deletion generations 2026-08-07 03:48:11 +08:00
hetaoBackend
bd964166f5 fix: persist remote upload deletion recovery 2026-08-07 03:25:59 +08:00
hetaoBackend
977be0d445 fix: recover upload deletes and revalidate remote sandboxes 2026-08-07 01:29:13 +08:00
hetaoBackend
5457406cc0 fix: close upload rollback and rollout races 2026-08-07 00:58:11 +08:00
hetaoBackend
8cddb8663a fix: harden upload mount and deletion contracts 2026-08-07 00:31:17 +08:00
hetaoBackend
9f03fa7563 fix: close upload upgrade compatibility gaps 2026-08-06 23:52:07 +08:00
hetaoBackend
ec7e5e5db1 fix: preserve upload edge-case compatibility 2026-08-06 23:26:24 +08:00
hetaoBackend
297bee0a70 fix: prevent upload lease deadlocks 2026-08-06 23:04:37 +08:00
hetaoBackend
96ba5dccd0 fix: coordinate upload aliases at commit 2026-08-06 22:14:24 +08:00
hetaoBackend
783920f75a fix: close upload rollback gaps 2026-08-06 22:01:42 +08:00
hetaoBackend
0082631e23 fix: synchronize every upload ingress 2026-08-06 21:44:08 +08:00
hetaoBackend
e75981a873 docs: clarify upload sandbox boundaries 2026-08-06 20:55:36 +08:00
hetaoBackend
a540ff46a9 docs: document upload generation leases 2026-08-06 20:24:02 +08:00
hetaoBackend
8a3cb391f6 docs: define collision-safe upload layout 2026-08-06 10:27:19 +08:00
Felix Wang
cd8825b0f0
docs(uploads): document current upload context (#4632) 2026-08-02 08:52:27 +08:00
阿泽
9c7cd4cad3
feat(sandbox): add thread data mount override for upload sync (#4536) 2026-07-28 23:41:14 +08:00
KiteEater
8939ccaed2
fix(uploads): enforce streaming upload limits in gateway (#2589)
* fix: enforce gateway upload limits

* fix: acquire sandbox before upload writes

* Fix upload limit config wiring

* Sanitize upload size error filenames

* test: call upload routes unwrapped

* fix: guard upload limits endpoint

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-05-01 20:19:30 +08:00
Hinotobi
80e210f5bb
[security] fix(uploads): require explicit opt-in for host-side document conversion (#2332)
* fix: disable host-side upload conversion by default

* fix: address PR review comments on upload conversion gate
2026-04-18 22:47:42 +08:00
DanielWalnut
76803b826f
refactor: split backend into harness (deerflow.*) and app (app.*) (#1131)
* refactor: extract shared utils to break harness→app cross-layer imports

Move _validate_skill_frontmatter to src/skills/validation.py and
CONVERTIBLE_EXTENSIONS + convert_file_to_markdown to src/utils/file_conversion.py.
This eliminates the two reverse dependencies from client.py (harness layer)
into gateway/routers/ (app layer), preparing for the harness/app package split.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: split backend/src into harness (deerflow.*) and app (app.*)

Physically split the monolithic backend/src/ package into two layers:

- **Harness** (`packages/harness/deerflow/`): publishable agent framework
  package with import prefix `deerflow.*`. Contains agents, sandbox, tools,
  models, MCP, skills, config, and all core infrastructure.

- **App** (`app/`): unpublished application code with import prefix `app.*`.
  Contains gateway (FastAPI REST API) and channels (IM integrations).

Key changes:
- Move 13 harness modules to packages/harness/deerflow/ via git mv
- Move gateway + channels to app/ via git mv
- Rename all imports: src.* → deerflow.* (harness) / app.* (app layer)
- Set up uv workspace with deerflow-harness as workspace member
- Update langgraph.json, config.example.yaml, all scripts, Docker files
- Add build-system (hatchling) to harness pyproject.toml
- Add PYTHONPATH=. to gateway startup commands for app.* resolution
- Update ruff.toml with known-first-party for import sorting
- Update all documentation to reflect new directory structure

Boundary rule enforced: harness code never imports from app.
All 429 tests pass. Lint clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add harness→app boundary check test and update docs

Add test_harness_boundary.py that scans all Python files in
packages/harness/deerflow/ and fails if any `from app.*` or
`import app.*` statement is found. This enforces the architectural
rule that the harness layer never depends on the app layer.

Update CLAUDE.md to document the harness/app split architecture,
import conventions, and the boundary enforcement test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add config versioning with auto-upgrade on startup

When config.example.yaml schema changes, developers' local config.yaml
files can silently become outdated. This adds a config_version field and
auto-upgrade mechanism so breaking changes (like src.* → deerflow.*
renames) are applied automatically before services start.

- Add config_version: 1 to config.example.yaml
- Add startup version check warning in AppConfig.from_file()
- Add scripts/config-upgrade.sh with migration registry for value replacements
- Add `make config-upgrade` target
- Auto-run config-upgrade in serve.sh and start-daemon.sh before starting services
- Add config error hints in service failure messages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix comments

* fix: update src.* import in test_sandbox_tools_security to deerflow.*

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: handle empty config and search parent dirs for config.example.yaml

Address Copilot review comments on PR #1131:
- Guard against yaml.safe_load() returning None for empty config files
- Search parent directories for config.example.yaml instead of only
  looking next to config.yaml, fixing detection in common setups

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: correct skills root path depth and config_version type coercion

- loader.py: fix get_skills_root_path() to use 5 parent levels (was 3)
  after harness split, file lives at packages/harness/deerflow/skills/
  so parent×3 resolved to backend/packages/harness/ instead of backend/
- app_config.py: coerce config_version to int() before comparison in
  _check_config_version() to prevent TypeError when YAML stores value
  as string (e.g. config_version: "1")
- tests: add regression tests for both fixes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: update test imports from src.* to deerflow.*/app.* after harness refactor

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 22:55:52 +08:00
Willem Jiang
8c6dd9e264
fix(uploads): persist thread uploads canonically and fail fast on upload errors (#943)
* fix(uploads): persist thread uploads canonically and fail fast on upload errors

 - write uploads to thread-scoped storage first to guarantee agent visibility
 - sync files to sandbox virtual path only for non-local sandboxes
 - fix markdown conversion flow to operate on canonical saved files and sync converted files when needed
 - prevent silent attachment upload failures in frontend submit flow (show error + abort submit)
 - add regression tests for local vs non-local upload behavior
 - update upload docs with thread-first persistence and troubleshooting notes

* Update frontend/src/core/threads/hooks.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix(uploads): reject "." and ".." filenames in upload sanitization (#944)

* Initial plan

* fix(uploads): reject '.' and '..' filenames in upload sanitization

Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-01 15:35:30 +08:00
hetao
f6a20a69e3 feat: implement file upload feature 2026-01-23 18:47:39 +08:00