deer-flow/backend/tests/test_project_documents_router.py
Zeren Wang a58ab484a6
feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash (#5443)
* feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash

Implements docs/superpowers/specs/2026-09-12-projects-mvp-phase2-design.md
(issue #5160, tracker #5129) in the slice order of the spec's §16.

Slices:
- A: ProjectsConfig + write-time 422 UTF-8 byte cap; PROJECT_CONTEXT_KEY
  admission pinning (both server-owned sets + worker hoist); latest-only
  request-scoped <project> block via DynamicContextMiddleware
  wrap_model_call/awrap_model_call (idempotent reassembly, reserved ID
  prefix + marker + provenance, never persisted); journal audit
  fingerprints; Instructions tab.
- B: ProjectDocumentRow + migration 0023; ProjectDocumentRepository with
  locked check-and-set; hash-qualified immutable shelf storage with
  Paths helpers; upload/list/content/delete-to-trash routes; project
  delete trashes the shelf in-transaction; request-scoped bounded
  <documents> index with honest count/shown + actionable overflow note;
  list_project_documents/read_project_document tools registered only on
  pinned runs; PAT allowlist + drift guards; blocking-IO anchors.
- C: shared thread-upload ingestion service (uploads router refactored to
  parity); POST from-thread with provenance; attach-to-thread with
  lock-staged copy (archived source allowed); read-only thread-files
  view with per-group truncation reporting.
- D: restore (restored/merged/not_found/no_target/content_missing; no
  file moves), purge (continuous row lock across unlink/delete/commit,
  retryable on FS errors), retention sweep (lazy + startup, 24h orphan
  guard, row-side reconciliation never deletes).
- E: Documents tab (shelf + conversation-files browser, provenance,
  archived banner, content-missing rows), /workspace/trash route,
  sidebar entry, composer attach handoff, i18n (en-US/zh-CN), e2e mocks
  + specs.

Review hardening folded in (10 rounds, all with tests):
- force active shelf content (HTML/XML family) to download; nosniff on
  artifact + content responses; unified unsandboxed-iframe PDF preview
  (fixes the pre-existing Chromium sandbox blank in the artifact viewer)
- scope document trash to the URL project under the document lock
- atomic no-overwrite filename reservation for ALL ingestion (seeded
  claims + os.link commit with suffix retry; same-name re-upload now
  unique-names instead of replacing); hidden staging only, no visible
  placeholders; lease cleanup on setup failure
- serialize conversion under the document lock with post-lock active
  revalidation; drain locked filesystem work on cancellation; preserve
  bytes when an insert's commit state is uncertain (including trashed
  rows)
- original-integrity checks before serving text or cached conversions;
  content_missing surfaced in list responses (UI reads the flag, no
  409-probe); downloads always serve original bytes
- bounded streaming document reads with cached char counts; shelf limits
  declared in middleware release identity
- thread-root confinement for from-thread sources; config fallback
  rejects fractional/infinite values; composer counts staged
  attachments; pending attachments persist until submission or removal;
  in-flight instruction/rename edits survive save refetches; shelf and
  trash pagination; conversation-file and thread-files pages stay
  subscribed to refetches

Docs: README/README_zh, backend API.md/ARCHITECTURE.md, AGENTS.md
contracts, config.example.yaml projects block.

Review follow-ups (head b4807477 → this revision):
- The trash retention sweep is split so repeated lazy triggers stay
  bounded: the indexed expiry purge still runs on every trigger
  (GET /api/trash/documents, POST /api/trash/purge) while the
  O(all rows + all files) reconciliation is throttled to one run per
  user per 15 minutes (process-local, per-user window). The startup
  sweep now runs as a background task instead of blocking gateway
  readiness, and shutdown awaits it (bounded).
- The export scrub (stripInternalMarkers) is fence- and indentation-aware
  like the render path, so a pasted, fenced <project>/<documents> snippet
  survives markdown export while real injected blocks (never fenced) are
  still removed. Fence regexes moved to a dependency-free leaf module to
  avoid the messages↔streamdown import cycle.
- The artifact viewer's PDF iframe no longer carries an added title
  attribute (the upstream e2e contract locates it via :not([title])), and
  the upstream artifact-preview spec now pins the new contract: PDFs
  render unsandboxed, images keep sandbox="".

* fix(projects): round-2 review — cancel an overrun trash sweep, restore the PDF frame title

- Shutdown cancelled only the shield around the background startup sweep,
  so an all-users reconciliation that outlived the 5s budget kept walking
  rows and files while the document repo and DB engine were disposed
  underneath it. The wait now lives in `_shutdown_startup_trash_sweep`,
  which cancels the task and drains it before worker exit: the shield
  keeps the wait bounded, the cancel makes it final (CancelledError lands
  at the sweep's next await, and `_run_startup_trash_sweep` only catches
  `Exception`, so nothing swallows it).
- The browser-preview iframe lost `title={getFileName(filepath)}` in the
  previous fix round, leaving the PDF frame without an accessible name
  while its siblings keep theirs. Restore it (WCAG frame titles), assert
  it in the DOM test, and anchor the e2e on `iframe[title="report.pdf"]`
  instead of `iframe:not([title])`.

* fix(projects): round-3 review — report the sweep's late finish, not a phantom cancel

`Task.cancel()` returns False when the sweep already finished inside the
window between the deadline firing and the cancel, so the shutdown log
claimed a cancellation that never happened. Branch on that outcome: the
warning stays for a real cancel, a late finish is logged at info, and both
paths still reap the task before worker exit.

* fix(projects): round-4 review — make Empty trash delete what it confirms

`POST /api/trash/purge` only ran the retention sweep, and the sweep's
candidate selection is age-gated, so a freshly trashed document survived
"Empty trash" even though the confirmation promises that every listed
document is permanently deleted. With one trashed row the route answered
`{"purged": 0}` and left it in place; `GET /api/trash/documents` sweeps
expired rows before listing, so the visible rows were normally ineligible
for the action by construction.

Empty trash now drives `purge_all_trashed`: the caller's trashed rows
(`list_all_trashed`, no age filter) each go through the same guarded,
row-locked `purge` as the single-document delete — bytes first, then the
row, in one transaction — so a row restored mid-flight is skipped instead of
force-deleted, and an unlink failure rolls that row back and answers 500 with
a retryable message. Retention expiry stays where it was: the sweep's
`purge_candidates` is now the only age-gated selection, and the lazy
retention sweep still runs on the listing and at startup.

Tests: the router suite replaces the retention-gated expectation with the
reviewer's repro (fresh row purged, bytes unlinked, shelf and other users'
trash untouched, a failing unlink stays retryable and 500); a blocking-I/O
anchor drives the new entry point through the offload; the mocked e2e covers
the action end to end; a new real-backend spec performs it against the real
gateway and re-reads `GET /api/trash/documents`. README, API, ARCHITECTURE
and the phase-2 design docs (en+zh) state the age-independent contract.
2026-09-16 18:46:18 +08:00

604 lines
29 KiB
Python

"""Integration tests for the project document shelf API (Phase-2 Slice B).
Upload → list → content → delete-to-trash round trip through the real
routers on a temp SQLite engine, fail-closed 404s for foreign/missing
resources on every route, the archived matrix (§8.4), filename byte
boundaries, dedup status codes, and project delete trashing the shelf.
"""
from __future__ import annotations
from types import SimpleNamespace
from typing import Any
from unittest.mock import MagicMock
import anyio
import pytest
from fastapi import FastAPI, Request, Response
from fastapi.testclient import TestClient
from starlette.middleware.base import BaseHTTPMiddleware
from app.gateway.authz import AuthContext, Permissions
from app.gateway.deps import get_config
from app.gateway.routers import project_documents, projects
from deerflow.persistence.engine import close_engine, get_session_factory, init_engine
from deerflow.persistence.projects import ProjectDocumentRepository, ProjectRepository
from deerflow.persistence.thread_meta import ThreadMetaRepository
from deerflow.runtime.user_context import reset_current_user, set_current_user
_STUB_PERMISSIONS: list[str] = [
Permissions.PROJECTS_READ,
Permissions.PROJECTS_WRITE,
Permissions.PROJECTS_DELETE,
]
_USER_HEADER = "x-test-user"
class _StubAuthMiddleware(BaseHTTPMiddleware):
"""Stamp a fake AuthContext and set the user ContextVar per request."""
async def dispatch(self, request: Request, call_next: Any) -> Response:
user_id = request.headers.get(_USER_HEADER, "user-a")
user = SimpleNamespace(id=user_id, system_role="user")
request.state.user = user
request.state.auth = AuthContext(user=user, permissions=list(_STUB_PERMISSIONS))
token = set_current_user(user)
try:
return await call_next(request)
finally:
reset_current_user(token)
async def _init_db(tmp_path) -> None:
await init_engine("sqlite", url=f"sqlite+aiosqlite:///{tmp_path / 'docs.db'}", sqlite_dir=str(tmp_path))
def _build_app(tmp_path, *, document_repo: Any = "default") -> FastAPI:
anyio.run(_init_db, tmp_path)
sf = get_session_factory()
app = FastAPI()
app.add_middleware(_StubAuthMiddleware)
app.state.project_repo = ProjectRepository(sf)
app.state.project_document_repo = ProjectDocumentRepository(sf) if document_repo == "default" else document_repo
app.state.thread_store = ThreadMetaRepository(sf)
cfg = MagicMock()
cfg.uploads = {}
app.dependency_overrides[get_config] = lambda: cfg
app.include_router(projects.router)
app.include_router(project_documents.router)
return app
def _as_user(user_id: str) -> dict[str, str]:
return {_USER_HEADER: user_id}
@pytest.fixture(autouse=True)
def _isolate(tmp_path, monkeypatch):
import deerflow.config.paths as paths_mod
monkeypatch.setenv("DEER_FLOW_HOME", str(tmp_path / "home"))
monkeypatch.setattr(paths_mod, "_paths", None)
yield
anyio.run(close_engine)
def _create_project(client: TestClient, name: str = "P", **kwargs) -> dict:
created = client.post("/api/projects", json={"name": name, **kwargs})
assert created.status_code == 201
return created.json()
def _upload(client: TestClient, project_id: str, name: str, data: bytes, **kwargs) -> Any:
return client.post(f"/api/projects/{project_id}/documents", files={"file": (name, data)}, **kwargs)
class TestRoundTrip:
def test_upload_list_content_delete_round_trip(self, tmp_path):
app = _build_app(tmp_path)
with TestClient(app) as client:
project = _create_project(client)
pid = project["id"]
created = _upload(client, pid, "notes.txt", b"hello shelf")
assert created.status_code == 201
body = created.json()
assert body["deduplicated"] is False
doc = body["document"]
assert doc["name"] == "notes.txt" and doc["size_bytes"] == 11
assert doc["source_kind"] == "upload"
listed = client.get(f"/api/projects/{pid}/documents")
assert listed.status_code == 200
payload = listed.json()
assert payload["total"] == 1
assert [d["id"] for d in payload["documents"]] == [doc["id"]]
content = client.get(f"/api/projects/{pid}/documents/{doc['id']}/content")
assert content.status_code == 200
assert content.text == "hello shelf"
assert "inline" in content.headers["content-disposition"]
assert content.headers["x-content-type-options"] == "nosniff"
download = client.get(f"/api/projects/{pid}/documents/{doc['id']}/content", params={"download": "true"})
assert download.status_code == 200
assert "attachment" in download.headers["content-disposition"]
assert download.headers["x-content-type-options"] == "nosniff"
deleted = client.delete(f"/api/projects/{pid}/documents/{doc['id']}")
assert deleted.status_code == 204
# Trashed rows are invisible to every endpoint.
assert client.get(f"/api/projects/{pid}/documents").json()["total"] == 0
assert client.get(f"/api/projects/{pid}/documents/{doc['id']}/content").status_code == 404
assert client.delete(f"/api/projects/{pid}/documents/{doc['id']}").status_code == 404
def test_dedup_hit_returns_200_with_the_first_writers_row(self, tmp_path):
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
first = _upload(client, pid, "first.txt", b"identical bytes")
assert first.status_code == 201
second = client.post(f"/api/projects/{pid}/documents", files={"file": ("second.txt", b"identical bytes")})
assert second.status_code == 200
body = second.json()
assert body["deduplicated"] is True
# First name wins (§10.9); no second row or file was published.
assert body["document"]["name"] == "first.txt"
assert body["document"]["id"] == first.json()["document"]["id"]
assert client.get(f"/api/projects/{pid}/documents").json()["total"] == 1
def test_reupload_after_trash_creates_a_fresh_row(self, tmp_path):
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
first = _upload(client, pid, "same.txt", b"bytes").json()["document"]
assert client.delete(f"/api/projects/{pid}/documents/{first['id']}").status_code == 204
second = _upload(client, pid, "same.txt", b"bytes")
assert second.status_code == 201
assert second.json()["document"]["id"] != first["id"]
def test_same_name_different_content_gets_distinct_ids(self, tmp_path):
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
a = _upload(client, pid, "report.pdf", b"version one").json()["document"]
b = _upload(client, pid, "report.pdf", b"version two").json()["document"]
assert a["id"] != b["id"]
names = {d["id"]: d["name"] for d in client.get(f"/api/projects/{pid}/documents").json()["documents"]}
assert names == {a["id"]: "report.pdf", b["id"]: "report.pdf"}
class TestListContentMissing:
"""The shelf list reports per-row read-time integrity (§8.3/§11): the
immutable ORIGINAL is the anchor — missing or size-mismatched ⇒
``content_missing: true``; a missing derived companion alone is fine."""
def test_missing_or_truncated_original_flags_content_missing(self, tmp_path):
from deerflow.config.paths import get_paths
from deerflow.projects.documents import original_file_path
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
_upload(client, pid, "healthy.txt", b"fine")
missing = _upload(client, pid, "missing.txt", b"was here").json()["document"]
truncated = _upload(client, pid, "truncated.txt", b"full bytes").json()["document"]
# No derived companion exists (conversion is lazy): the original
# alone decides integrity.
_upload(client, pid, "no-companion.txt", b"original only")
async def _damage() -> None:
repo = app.state.project_document_repo
paths = get_paths()
row = await repo.get(missing["id"], user_id="user-a")
original_file_path(paths, user_id="user-a", row=row).unlink()
row = await repo.get(truncated["id"], user_id="user-a")
original_file_path(paths, user_id="user-a", row=row).write_bytes(b"cut")
anyio.run(_damage)
body = client.get(f"/api/projects/{pid}/documents").json()
by_name = {d["name"]: d for d in body["documents"]}
assert by_name["healthy.txt"]["content_missing"] is False
assert by_name["missing.txt"]["content_missing"] is True
assert by_name["truncated.txt"]["content_missing"] is True
assert by_name["no-companion.txt"]["content_missing"] is False
class TestUploadValidation:
def test_filename_255_utf8_bytes_accepted_256_rejected(self, tmp_path):
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
# 252 ASCII chars + one 3-byte CJK char = 255 UTF-8 bytes exactly.
name_255 = "a" * 252 + ""
assert len(name_255.encode("utf-8")) == 255
ok = _upload(client, pid, name_255, b"x")
assert ok.status_code == 201
assert ok.json()["document"]["name"] == name_255
name_256 = "a" * 253 + ""
assert len(name_256.encode("utf-8")) == 256
rejected = _upload(client, pid, name_256, b"x")
assert rejected.status_code == 400
def test_explicit_name_field_is_validated_and_wins(self, tmp_path):
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
renamed = client.post(f"/api/projects/{pid}/documents", files={"file": ("raw.txt", b"x")}, data={"name": "display.txt"})
assert renamed.status_code == 201
assert renamed.json()["document"]["name"] == "display.txt"
# An empty name field is indistinguishable from omission on the
# wire and falls back to the multipart filename; whitespace-only
# and separator-bearing names are rejected server-side.
for bad in (" ", "a/b.txt", "a\\b.txt", ".."):
rejected = client.post(f"/api/projects/{pid}/documents", files={"file": ("raw.txt", b"x")}, data={"name": bad})
assert rejected.status_code == 400, bad
def test_empty_file_is_400(self, tmp_path):
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
assert _upload(client, pid, "empty.txt", b"").status_code == 400
def test_pagination_bounds_follow_the_422_convention(self, tmp_path):
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
assert client.get(f"/api/projects/{pid}/documents", params={"limit": 0}).status_code == 422
assert client.get(f"/api/projects/{pid}/documents", params={"limit": 1001}).status_code == 422
assert client.get(f"/api/projects/{pid}/documents", params={"offset": -1}).status_code == 422
assert client.get(f"/api/projects/{pid}/documents", params={"limit": 1, "offset": 0}).status_code == 200
def test_content_missing_is_an_explicit_error(self, tmp_path):
from deerflow.projects.documents import original_file_path
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
doc = _upload(client, pid, "lost.txt", b"was here").json()["document"]
async def _remove() -> None:
token = set_current_user(SimpleNamespace(id="user-a"))
try:
row = await app.state.project_document_repo.get(doc["id"])
original_file_path(__import__("deerflow.config.paths", fromlist=["get_paths"]).get_paths(), user_id="user-a", row=row).unlink()
finally:
reset_current_user(token)
anyio.run(_remove)
response = client.get(f"/api/projects/{pid}/documents/{doc['id']}/content")
assert response.status_code == 409
assert "content_missing" in response.json()["detail"]
class TestFailClosed:
def test_404_for_missing_or_foreign_resources_on_every_route(self, tmp_path):
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
doc = _upload(client, pid, "a.txt", b"a").json()["document"]
# Missing project (the POST carries its required multipart file).
assert client.get("/api/projects/nope/documents").status_code == 404
assert client.post("/api/projects/nope/documents", files={"file": ("a.txt", b"a")}).status_code == 404
assert client.get("/api/projects/nope/documents/x/content").status_code == 404
assert client.delete("/api/projects/nope/documents/x").status_code == 404
# Missing document.
assert client.get(f"/api/projects/{pid}/documents/nope/content").status_code == 404
assert client.delete(f"/api/projects/{pid}/documents/nope").status_code == 404
# Foreign user sees nothing, anywhere.
foreign = _as_user("user-b")
assert client.get(f"/api/projects/{pid}/documents", headers=foreign).status_code == 404
assert _upload(client, pid, "b.txt", b"b", headers=foreign).status_code == 404
assert client.get(f"/api/projects/{pid}/documents/{doc['id']}/content", headers=foreign).status_code == 404
assert client.delete(f"/api/projects/{pid}/documents/{doc['id']}", headers=foreign).status_code == 404
def test_503_when_document_repo_unavailable(self, tmp_path):
app = _build_app(tmp_path, document_repo=None)
with TestClient(app) as client:
pid = _create_project(client)["id"]
assert client.get(f"/api/projects/{pid}/documents").status_code == 503
assert _upload(client, pid, "a.txt", b"a").status_code == 503
assert client.get(f"/api/projects/{pid}/documents/x/content").status_code == 503
assert client.delete(f"/api/projects/{pid}/documents/x").status_code == 503
class TestArchivedMatrix:
def test_archived_shelf_reads_work_but_mutations_are_404(self, tmp_path):
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
doc = _upload(client, pid, "a.txt", b"alpha").json()["document"]
assert client.post(f"/api/projects/{pid}/archive").status_code == 200
# Reads stay available (§8.4).
assert client.get(f"/api/projects/{pid}/documents").json()["total"] == 1
assert client.get(f"/api/projects/{pid}/documents/{doc['id']}/content").status_code == 200
# Mutations fail closed.
assert _upload(client, pid, "b.txt", b"b").status_code == 404
assert client.delete(f"/api/projects/{pid}/documents/{doc['id']}").status_code == 404
# Restore re-opens the shelf.
assert client.post(f"/api/projects/{pid}/restore").status_code == 200
assert _upload(client, pid, "b.txt", b"b").status_code == 201
assert client.delete(f"/api/projects/{pid}/documents/{doc['id']}").status_code == 204
class TestProjectDeleteTrashesShelf:
def test_delete_project_moves_shelf_to_trash_in_same_transaction(self, tmp_path):
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client, name="Doomed")["id"]
a = _upload(client, pid, "a.txt", b"a").json()["document"]
b = _upload(client, pid, "b.txt", b"b").json()["document"]
assert client.delete(f"/api/projects/{pid}").status_code == 204
async def _check() -> list[dict]:
token = set_current_user(SimpleNamespace(id="user-a"))
try:
repo = app.state.project_document_repo
out = []
for doc_id in (a["id"], b["id"]):
out.append(await repo.get(doc_id, include_trashed=True))
return out
finally:
reset_current_user(token)
rows = anyio.run(_check)
for row in rows:
assert row is not None and row["trashed_at"]
assert row["trash_origin"] == {"project_id": pid, "project_name": "Doomed"}
# No active row may reference the deleted project.
assert client.get(f"/api/projects/{pid}/documents").status_code == 404
class TestActiveContentForcedToDownload:
"""Active content on the app origin would run script, so it is always an
attachment (same semantics as the artifacts router) — even with
``download=false`` — while passive text and the converted-markdown
companion stay inline."""
def test_html_served_as_attachment_with_html_media_type(self, tmp_path):
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
doc = _upload(client, pid, "page.html", b"<script>alert(1)</script>").json()["document"]
for params in ({}, {"download": "false"}, {"download": "true"}):
response = client.get(f"/api/projects/{pid}/documents/{doc['id']}/content", params=params)
assert response.status_code == 200
assert response.headers["content-type"].startswith("text/html")
assert "attachment" in response.headers["content-disposition"]
def test_xml_family_types_served_as_attachments(self, tmp_path):
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
# ``+xml`` subtypes count as active content (SVG carries script;
# XHTML is namespaced HTML).
for name, data in (
("icon.svg", b'<svg xmlns="http://www.w3.org/2000/svg"/>'),
("page.xhtml", b'<html xmlns="http://www.w3.org/1999/xhtml"/>'),
("feed.xml", b"<?xml version='1.0'?><rss/>"),
):
doc = _upload(client, pid, name, data).json()["document"]
response = client.get(f"/api/projects/{pid}/documents/{doc['id']}/content")
assert response.status_code == 200, name
assert "attachment" in response.headers["content-disposition"], name
def test_passive_text_stays_inline(self, tmp_path):
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
doc = _upload(client, pid, "notes.txt", b"hello").json()["document"]
response = client.get(f"/api/projects/{pid}/documents/{doc['id']}/content")
assert response.status_code == 200
assert response.headers["content-type"].startswith("text/plain")
assert "inline" in response.headers["content-disposition"]
download = client.get(f"/api/projects/{pid}/documents/{doc['id']}/content", params={"download": "true"})
assert "attachment" in download.headers["content-disposition"]
def test_converted_markdown_companion_stays_inline(self, tmp_path):
from deerflow.config.paths import get_paths
from deerflow.projects.documents import converted_markdown_path
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
doc = _upload(client, pid, "report.html", b"<p>hi</p>").json()["document"]
async def _row() -> dict:
row = await app.state.project_document_repo.get(doc["id"], user_id="user-a")
assert row is not None
return row
derived = converted_markdown_path(get_paths(), user_id="user-a", row=anyio.run(_row))
derived.parent.mkdir(parents=True, exist_ok=True)
derived.write_text("# converted")
response = client.get(f"/api/projects/{pid}/documents/{doc['id']}/content")
assert response.status_code == 200
assert response.headers["content-type"].startswith("text/markdown")
assert "inline" in response.headers["content-disposition"]
class TestInlineViewableBinaries:
"""Browser-viewable binaries (PDF, image, audio, video) serve inline with
``download=false`` — the sandboxed preview iframe navigates to this
endpoint — while ``download=true`` and active content stay attachments."""
def test_pdf_image_video_are_inline_when_not_downloading(self, tmp_path):
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
for name, data, media_type in (
("report.pdf", b"%PDF-1.4\nfake", "application/pdf"),
("photo.png", b"\x89PNG\r\n\x1a\nfake", "image/png"),
("clip.mp4", b"\x00\x00\x00\x18ftypmp42", "video/mp4"),
):
doc = _upload(client, pid, name, data).json()["document"]
response = client.get(f"/api/projects/{pid}/documents/{doc['id']}/content")
assert response.status_code == 200, name
assert response.headers["content-type"].startswith(media_type), name
assert "inline" in response.headers["content-disposition"], name
download = client.get(f"/api/projects/{pid}/documents/{doc['id']}/content", params={"download": "true"})
assert "attachment" in download.headers["content-disposition"], name
class TestContentEndpointOriginalIntegrity:
"""The derived companion is served only while its immutable ORIGINAL
validates (existence AND recorded size, §6.2/§8.3) — a truncated original
behind a cached conversion is ``content_missing``, never a healthy
preview."""
def test_cached_derived_with_truncated_original_is_content_missing(self, tmp_path):
from deerflow.config.paths import get_paths
from deerflow.projects.documents import converted_markdown_path, original_file_path
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
doc = _upload(client, pid, "report.html", b"<p>hi</p>").json()["document"]
async def _plant_and_truncate() -> None:
row = await app.state.project_document_repo.get(doc["id"], user_id="user-a")
assert row is not None
paths = get_paths()
derived = converted_markdown_path(paths, user_id="user-a", row=row)
derived.parent.mkdir(parents=True, exist_ok=True)
derived.write_text("# converted")
original_file_path(paths, user_id="user-a", row=row).write_bytes(b"<p")
anyio.run(_plant_and_truncate)
response = client.get(f"/api/projects/{pid}/documents/{doc['id']}/content")
assert response.status_code == 409
assert "content_missing" in response.json()["detail"]
def test_cached_derived_with_valid_original_is_served(self, tmp_path):
from deerflow.config.paths import get_paths
from deerflow.projects.documents import converted_markdown_path
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
doc = _upload(client, pid, "report.html", b"<p>hi</p>").json()["document"]
async def _plant() -> None:
row = await app.state.project_document_repo.get(doc["id"], user_id="user-a")
assert row is not None
derived = converted_markdown_path(get_paths(), user_id="user-a", row=row)
derived.parent.mkdir(parents=True, exist_ok=True)
derived.write_text("# converted")
anyio.run(_plant)
response = client.get(f"/api/projects/{pid}/documents/{doc['id']}/content")
assert response.status_code == 200
assert response.headers["content-type"].startswith("text/markdown")
assert "inline" in response.headers["content-disposition"]
class TestDownloadServesOriginal:
"""The converted-markdown companion is preview-only: ``download=true``
always serves the immutable ORIGINAL bytes under the original filename,
so a prior agent read never changes the download's format."""
def test_download_true_serves_original_bytes_not_the_conversion(self, tmp_path):
from deerflow.config.paths import get_paths
from deerflow.projects.documents import converted_markdown_path
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
doc = _upload(client, pid, "report.pdf", b"%PDF-1.4 original").json()["document"]
async def _plant() -> None:
row = await app.state.project_document_repo.get(doc["id"], user_id="user-a")
assert row is not None
derived = converted_markdown_path(get_paths(), user_id="user-a", row=row)
derived.parent.mkdir(parents=True, exist_ok=True)
derived.write_text("# converted")
anyio.run(_plant)
response = client.get(f"/api/projects/{pid}/documents/{doc['id']}/content", params={"download": "true"})
assert response.status_code == 200
assert response.content == b"%PDF-1.4 original"
disposition = response.headers["content-disposition"]
assert "attachment" in disposition
assert "report.pdf" in disposition
assert "report.pdf.md" not in disposition
def test_download_false_on_convertible_serves_converted_markdown_inline(self, tmp_path):
from deerflow.config.paths import get_paths
from deerflow.projects.documents import converted_markdown_path
app = _build_app(tmp_path)
with TestClient(app) as client:
pid = _create_project(client)["id"]
doc = _upload(client, pid, "report.pdf", b"%PDF-1.4 original").json()["document"]
async def _plant() -> None:
row = await app.state.project_document_repo.get(doc["id"], user_id="user-a")
assert row is not None
derived = converted_markdown_path(get_paths(), user_id="user-a", row=row)
derived.parent.mkdir(parents=True, exist_ok=True)
derived.write_text("# converted")
anyio.run(_plant)
response = client.get(f"/api/projects/{pid}/documents/{doc['id']}/content")
assert response.status_code == 200
assert response.text == "# converted"
assert response.headers["content-type"].startswith("text/markdown")
assert "inline" in response.headers["content-disposition"]
class TestDeleteScopesToTheUrlProject:
"""``trash`` predicates on the URL project (§6.1's guarded UPDATE): a
document of a sibling project is indistinguishable from missing even when
the caller owns both shelves."""
def test_cross_project_delete_is_404_and_leaves_the_document_active(self, tmp_path):
app = _build_app(tmp_path)
with TestClient(app) as client:
project_a = _create_project(client, name="A")
project_b = _create_project(client, name="B")
doc = _upload(client, project_b["id"], "b.txt", b"b").json()["document"]
# Both projects owned: the URL project must still match the row.
assert client.delete(f"/api/projects/{project_a['id']}/documents/{doc['id']}").status_code == 404
assert client.get(f"/api/projects/{project_b['id']}/documents/{doc['id']}/content").status_code == 200
# Same-project delete still works.
assert client.delete(f"/api/projects/{project_b['id']}/documents/{doc['id']}").status_code == 204
assert client.get(f"/api/projects/{project_b['id']}/documents/{doc['id']}/content").status_code == 404
def test_restored_row_is_not_trashable_via_the_old_project_url(self, tmp_path):
app = _build_app(tmp_path)
with TestClient(app) as client:
project_a = _create_project(client, name="A")
project_b = _create_project(client, name="B")
doc = _upload(client, project_a["id"], "a.txt", b"a").json()["document"]
assert client.delete(f"/api/projects/{project_a['id']}/documents/{doc['id']}").status_code == 204
async def _restore_into_b() -> None:
outcome, _ = await app.state.project_document_repo.restore(doc["id"], target_project_id=project_b["id"], user_id="user-a")
assert outcome == "restored"
anyio.run(_restore_into_b)
# The row now lives on B's shelf: A's URL fails closed and the
# document stays active under B.
assert client.delete(f"/api/projects/{project_a['id']}/documents/{doc['id']}").status_code == 404
assert client.get(f"/api/projects/{project_b['id']}/documents/{doc['id']}/content").status_code == 200
assert client.delete(f"/api/projects/{project_b['id']}/documents/{doc['id']}").status_code == 204