* fix(skills): support basename data-analysis exports
Use the current directory when export output has no parent directory.
* test(skills): cover data-analysis export paths
Exercise real DuckDB queries, filesystem exports and cached CLI execution across CSV, JSON and Markdown.
* docs(skills): record export path regression guidance
Document basename and nested output expectations and the regression test command.
* docs(skills): explain data-analysis export paths
Clarify bare filenames, relative parent directories and absolute output paths.
* docs: note supported data-analysis export paths
Add a concise README explanation while retaining existing upstream documentation.
* docs(skills): clarify export overwrite behavior
Document that existing export targets are overwritten without warning, as suggested in PR #7132. Verified real CSV, JSON and Markdown overwrites; 27 export regression tests pass.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(chart-visualization): wire generate_spreadsheet and fail CLI nonzero
SKILL.md already selects generate_spreadsheet and the reference file
documents it, but CHART_TYPE_MAP had no entry, so the CLI reported
Unknown tool. Map it to the AntV spreadsheet type and exit non-zero
after any unknown tool, missing tool, or HTTP failure. Successful chart
URLs stay on stdout.
Co-authored-by: wannT-1 <200890097+wannT-1@users.noreply.github.com>
* docs(agents): fold chart CLI failures into the shared stderr/stdout note
The sandbox guidance chain was over the AG002 hard limit. Share one
nonzero-exit rule for video, podcast, and chart-visualization CLIs.
Co-authored-by: wannT-1 <wannT-1@users.noreply.github.com>
---------
Co-authored-by: wannT-1 <200890097+wannT-1@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: wannT-1 <wannT-1@users.noreply.github.com>
* fix(skills): preserve transparency in composed PowerPoint slides
* ci(skills): run public skill regressions
* ci: remove public skill test workflow
* docs: move skill test commands out of inherited guidance
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(skills): validate music audio before opening output files
* fix(skills): reject non-MP3 music responses before writing
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
`get_readme` and `get_file_content` answered every failure with a string --
`[README not found: HTTP 404]`, `[File not found: HTTP 404]` -- while `main()`
prints any string result on stdout and exits 0. A rate-limited run, a missing
repository and a real README therefore arrived on the same channel with the same
exit code, so a caller reading stdout could not tell a failure from a body.
Both readers now raise `GitHubAPIError` naming the operation, the target and the
underlying HTTP error. The CLI's existing handler already writes an exception to
stderr and exits 1, so a failure leaves the success channel empty and the status
stays visible; a successful fetch is unchanged.
Pins both transports (requests and the urllib fallback), 404 against 403 so a
rate limit cannot read as a missing repository, the file reader's path, and the
CLI contract: non-zero exit, empty stdout, status on stderr.
* fix(skills): preserve Markdown table cells in data-analysis exports
* docs(skills): document Markdown cell escaping contract
Clarify handling of Markdown exports and escaping rules.
* docs: describe data-analysis Markdown export escaping
Clarify data-analysis skill features and behavior in README.
* test(skills): cover Markdown export headers and multiline cells
Add a test for markdown export functionality in data analysis.
* fix(skill-creator): tolerate non-object timing measurements
Treat optional timing JSON values that are not objects as missing measurements so they cannot abort benchmark aggregation. Preserve grading durations, character-count fallback and recorded zeros; add regression coverage for both layouts and subsequent runs.
* fix(skill-creator): warn and validate timing measurements
* fix(skill-creator): tolerate unreadable timing files
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(github-research): resolve the tree against the repository's default branch
The tree command asked for "main" and, when that was missing, for "master".
A repository whose default branch is neither gets a name-guessing fallback, and
when the stale "master" still resolves GitHub returns it happily: the agent is
handed the tree of an abandoned branch and reports it as the current structure.
Ask for HEAD instead, which GitHub resolves to the repository's own default
branch, so no candidate names are guessed and no extra request is added.
* test(github-research): clarify stale branch tree comment
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(skill-creator): preserve recorded token counts in benchmarks
Read timing.json token measurements independently of grading duration and retain recorded zero values. Keep the existing character-count fallback for missing measurements and add benchmark/report regression coverage.
* fix(skillscan): distinguish token metrics from credentials
Recognize typed token counts, numeric benchmark summaries, and signed deltas so skill-creator documentation passes deterministic review. Preserve credential names, string values, duplicate-key detection, and cloud-token findings; add regression coverage and document the boundary.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Preserve filename delimiters and literal percent escapes before either HTTP transport builds a Contents API request. Add offline regression coverage for requests and the urllib fallback.
* test: fix five Windows-host failures across sandbox, read_file, lark shim, and extension checks
Windows dev hosts currently fail five backend test cases that CI (ubuntu)
never exercises:
- test_local_sandbox_command_timeout: the fixture forces a bare /bin/sh
shell and the Windows dispatch spawns it directly (WinError 2), and
make(1) is not a given on Windows. Stub both runners; the contract under
test is the timeout marker assembly, not the runner.
- test_local_sandbox_provider_mounts: the Popen-capture mock delegated to
the real spawn, which cannot execute a POSIX shell path on Windows.
Capture at the runner seam instead and assert the rewritten path
shape-agnostically (the Windows runners pass the command string, not a
[shell, -c, command] list).
- test_read_file_tool_binary: text-mode write_text leaks CRLF into the
fixture on Windows; write the mount fixture as bytes (#6042 follow-up).
- test_lark_shim_atomic: skip the exact 0o755/0o644 assertions on Windows
hosts, whose stat mode approximates POSIX bits (#6257 pattern).
- test_extension_check_script: zipfile normalizes os.sep in entry names on
Windows reads, so a writestr-built fixture can never carry a verbatim
backslash entry there. Inject the hostile name into namelist unless the
archive already stores it, pinning the guard contract identically.
On a Windows 11 host these five files go from five failed cases to green
(102 passed, ruff check/format clean). Two more Windows-host failures
(skill-review NUL probe, jina shared-deadline classification) need deeper
root-causing and are left for follow-ups.
* test(skill-review): review the committed skill tree, not local residue
The video-generation review test scans the live directory, so untracked
build residue inside skills/public/video-generation fails it: once bytecode
caching drops a scripts/__pycache__ directory into the tree (importing the
skill script writes one), skillscan's package-undecodable-script finding is
correct and the test can never pass again until the residue is deleted by
hand. Reproduced on Windows: a planted scripts/__pycache__ pyc fails the
test; deleting it makes it pass.
- Materialize the tracked skill tree (git ls-files) into a temp dir and
review that, so the test reviews exactly the committed package.
- tests/skills/skill_loader.py disables bytecode caching around
exec_module, so the root skill suite stops polluting the committed tree.
Follows up on #6786 (item 6).
* docs(test): state the argv-list contract on both sandbox runners
The Windows dispatch also passes the resolved command as an argv list, so
the string fallback was unreachable on every platform.
---------
Co-authored-by: xiaodu55 <xiaodu55@users.noreply.github.com>
* fix(video): forward --aspect-ratio into the Gemini Veo request
The video-generation skill accepts --aspect-ratio and passes it into
generate_video(), but the Gemini branch drops the value:
_generate_video_gemini has no aspect_ratio parameter and builds the
predictLongRunning body with only instances, so every Veo request runs
at the default ratio regardless of the CLI flag. Forward the value as
parameters.aspectRatio and cover it with a monkeypatch regression test
that captures the outgoing request body.
* fix(skills): avoid false credential findings in video generation
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* docs(spec): MiniMax integration for generation skills + new music skill
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(plan): MiniMax generation providers implementation plan
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(skills): add importlib loader + FakeResp for skill tests
* test(skills): register loaded module in sys.modules; raise requests.HTTPError in FakeResp
* feat(image-generation): add MiniMax provider with env auto-detect
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(image-generation): guard unknown provider, derive ref MIME, strengthen tests
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(video-generation): add MiniMax provider with async poll/download
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(video-generation): surface base_resp errors while polling; add timeout test
* feat(podcast-generation): add MiniMax t2a_v2 provider with env auto-detect
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(podcast-generation): restore TTS credential guard; add volcengine + voice tests
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(music-generation): new MiniMax music skill via skill-creator
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(music-generation): treat empty lyrics as absent; test no-audio-data path
* refactor(skills): add request timeouts to MiniMax network calls
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Potential fix for pull request finding 'Explicit returns mixed with implicit (fall through) returns'
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
* fix(models): strip inconsistent user-message names for MiniMax chat
DeerFlow middlewares tag user messages with provenance names (user-input, summary, loop_warning); langchain serializes them into the OpenAI-compatible payload and MiniMax rejects mismatched user-message names with "user name must be consistent (2013)". PatchedChatMiniMax now drops the per-message name from user-role messages. Point the config.example MiniMax models at PatchedChatMiniMax so they also get reasoning_content mapping.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(image-generation): MiniMax sends JSON prompt field, guard 1500-char limit
MiniMax image-01 takes one text string capped at 1500 chars, but the skill was sending the whole structured JSON. The MiniMax provider now extracts the JSON `prompt` field (relying on prompt_optimizer to expand it) and fails fast with a clear error before calling the API when that field exceeds 1500 chars. Authoring stays provider-agnostic; Gemini still receives the full JSON.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(podcast-generation): per-provider TTS concurrency and retry/backoff
Each TTS provider owns its concurrency internally — MiniMax runs single-threaded to reduce rate-limit failures, Volcengine keeps 4 workers — with automatic retry and backoff on transient HTTP and base_resp errors. No caller-facing concurrency knob.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(skills): address Copilot review comments on generation skills
- video: add raise_for_status + timeout to the Gemini download/POST/poll calls so non-2xx responses surface as clear HTTP errors instead of JSON/KeyError or hangs
- video: check the task Fail status before the generic base_resp check so the failure keeps its task_id context
- video/image: create the output file parent directory before writing (matching music-generation) so nested output paths do not raise FileNotFoundError
- music: require a non-empty prompt and fail fast with ValueError instead of sending an empty prompt to the API
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(scripts): reclaim dev ports across worktrees in make stop/dev
All deer-flow worktrees (main checkout + linked worktrees) hardcode the same dev ports (8001/3000/2026), so a service started from any worktree must be reclaimable from another. stop_all now resolves the set of worktree roots (DEERFLOW_ROOTS) and treats a process as deer-flow-owned when its open files live under any of them. It also force-kills survivors on 2026 alongside 8001/3000, fixing `make dev` aborting on the nginx port preflight when a prior nginx lingered on 2026.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(view-image): hide the injected image-context message from the UI
ViewImageMiddleware injects a HumanMessage (text + base64 images) so the vision model can see viewed images, but it was the only internal injector that set neither hide_from_ui nor a hidden name, so it leaked into the chat UI (and IM channels) as a user bubble reading "Here are the images you've viewed:". Mark it with additional_kwargs={"hide_from_ui": True}, matching todo/dynamic_context injections, which the frontend isHiddenFromUIMessage and the channel sender already honor. The model still receives the full content.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(minimax): mark M2.7 models as text-only (no vision)
MiniMax M2.7 / M2.7-highspeed do not support vision; only M3 does. The
provider config asserted vision support for M2.7 in four places.
- config.example.yaml: 4 M2.7 entries -> supports_vision: false
- backend/docs/CONFIGURATION.md: M2.7 + highspeed -> supports_vision: false
- wizard: add LLMProvider.model_vision_overrides + extra_config_for() so
selecting an M2.7 model writes supports_vision: false while M3 (default)
keeps vision; wire it through setup_wizard.py
- tests: M2.7-highspeed fixture -> supports_vision=False; add
test_minimax_vision_is_per_model
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
* feat: add citation support in research report block and markdown
- Enhanced ResearchReportBlock to fetch citations based on researchId and pass them to the Markdown component.
- Introduced CitationLink component to display citation metadata on hover for links in markdown.
- Implemented CitationCard and CitationList components for displaying citation details and lists.
- Updated Markdown component to handle citation links and inline citations.
- Created HoverCard component for displaying citation information in a tooltip-like manner.
- Modified store to manage citations, including setting and retrieving citations for ongoing research.
- Added CitationsEvent type to handle citations in chat events and updated Message type to include citations.
* fix(log): Enable the logging level when enabling the DEBUG environment variable (#793)
* fix(frontend): render all tool calls in the frontend #796 (#797)
* build(deps): bump jspdf from 3.0.4 to 4.0.0 in /web (#798)
Bumps [jspdf](https://github.com/parallax/jsPDF) from 3.0.4 to 4.0.0.
- [Release notes](https://github.com/parallax/jsPDF/releases)
- [Changelog](https://github.com/parallax/jsPDF/blob/master/RELEASE.md)
- [Commits](https://github.com/parallax/jsPDF/compare/v3.0.4...v4.0.0)
---
updated-dependencies:
- dependency-name: jspdf
dependency-version: 4.0.0
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix(frontend):added the display of the 'analyst' message #800 (#801)
* fix: migrate from deprecated create_react_agent to langchain.agents.create_agent (#802)
* fix: migrate from deprecated create_react_agent to langchain.agents.create_agent
Fixes#799
- Replace deprecated langgraph.prebuilt.create_react_agent with
langchain.agents.create_agent (LangGraph 1.0 migration)
- Add DynamicPromptMiddleware to handle dynamic prompt templates
(replaces the 'prompt' callable parameter)
- Add PreModelHookMiddleware to handle pre-model hooks
(replaces the 'pre_model_hook' parameter)
- Update AgentState import from langchain.agents in template.py
- Update tests to use the new API
* fix:update the code with review comments
* fix: Add runtime parameter to compress_messages method(#803)
* fix: Add runtime parameter to compress_messages method(#803)
The compress_messages method was being called by PreModelHookMiddleware
with both state and runtime parameters, but only accepted state parameter.
This caused a TypeError when the middleware executed the pre_model_hook.
Added optional runtime parameter to compress_messages signature to match
the expected interface while maintaining backward compatibility.
* Update the code with the review comments
* fix: Refactor citation handling and add comprehensive tests for citation features
* refactor: Clean up imports and formatting across citation modules
* fix: Add monkeypatch to clear AGENT_RECURSION_LIMIT in recursion limit tests
* feat: Enhance citation link handling in Markdown component
* fix: Exclude citations from finish reason handling in mergeMessage function
* fix(nodes): update message handling
* fix(citations): improve citation extraction and handling in event processing
* feat(citations): enhance citation extraction and handling with improved merging and normalization
* fix(reporter): update citation formatting instructions for clarity and consistency
* fix(reporter): prioritize using Markdown tables for data presentation and comparison
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: LoftyComet <1277173875@qq。>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This PR addresses token limit issues when web_search is enabled with include_raw_content by implementing a two-pronged approach: changing the default behavior to exclude raw content and adding compression logic for when raw content is included.
* feat: Implement DeerFlow API server with chat streaming, Langgraph orchestration, and various content generation capabilities.
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* - Use MongoDB `$push` with `$each` to append new messages to existing threads
- Use PostgreSQL jsonb concatenation operator to merge messages instead of overwriting
- Update comments to reflect append behavior in both database implementations
* fix: updated the unit tests with the recent changes
---------
Co-authored-by: Bink <992359580@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: YikB <54528024+Bin1783@users.noreply.github.com>
Fixes a critical bug in the from_runnable_config() method where falsy values (like False, 0, and empty strings) were being incorrectly filtered out, causing configuration fields to revert to their default values. The fix changes the filter condition from if v to if v is not None, ensuring only None values are skipped.
* fix(config): Add support for MCP server configuration parameters
* refact: rename the sse_readtimeout to sse_read_timeout
* update the code with review comments
* update the MCP document for the latest change
* fix: Add runtime parameter to compress_messages method(#803)
The compress_messages method was being called by PreModelHookMiddleware
with both state and runtime parameters, but only accepted state parameter.
This caused a TypeError when the middleware executed the pre_model_hook.
Added optional runtime parameter to compress_messages signature to match
the expected interface while maintaining backward compatibility.
* Update the code with the review comments
* fix: migrate from deprecated create_react_agent to langchain.agents.create_agent
Fixes#799
- Replace deprecated langgraph.prebuilt.create_react_agent with
langchain.agents.create_agent (LangGraph 1.0 migration)
- Add DynamicPromptMiddleware to handle dynamic prompt templates
(replaces the 'prompt' callable parameter)
- Add PreModelHookMiddleware to handle pre-model hooks
(replaces the 'pre_model_hook' parameter)
- Update AgentState import from langchain.agents in template.py
- Update tests to use the new API
* fix:update the code with review comments
* fix(podcast): add fallback for models without json_object support (#747)
Models like Kimi K2 don't support response_format.type: json_object.
Add try-except to fall back to regular prompting with JSON parsing
when BadRequestError mentions json_object not supported.
- Add fallback to prompting + repair_json_output parsing
- Re-raise other BadRequestError types
- Add unit tests for script_writer_node with 100% coverage
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fixes: the unit test error of test_script_writer_node.py
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat(eval): add report quality evaluation module
Addresses issue #773 - How to evaluate generated report quality objectively.
This module provides two evaluation approaches:
1. Automated metrics (no LLM required):
- Citation count and source diversity
- Word count compliance per report style
- Section structure validation
- Image inclusion tracking
2. LLM-as-Judge evaluation:
- Factual accuracy scoring
- Completeness assessment
- Coherence evaluation
- Relevance and citation quality checks
The combined evaluator provides a final score (1-10) and letter grade (A+ to F).
Files added:
- src/eval/__init__.py
- src/eval/metrics.py
- src/eval/llm_judge.py
- src/eval/evaluator.py
- tests/unit/eval/test_metrics.py
- tests/unit/eval/test_evaluator.py
* feat(eval): integrate report evaluation with web UI
This commit adds the web UI integration for the evaluation module:
Backend:
- Add EvaluateReportRequest/Response models in src/server/eval_request.py
- Add /api/report/evaluate endpoint to src/server/app.py
Frontend:
- Add evaluateReport API function in web/src/core/api/evaluate.ts
- Create EvaluationDialog component with grade badge, metrics display,
and optional LLM deep evaluation
- Add evaluation button (graduation cap icon) to research-block.tsx toolbar
- Add i18n translations for English and Chinese
The evaluation UI allows users to:
1. View quick metrics-only evaluation (instant)
2. Optionally run deep LLM-based evaluation for detailed analysis
3. See grade (A+ to F), score (1-10), and metric breakdown
* feat(eval): improve evaluation reliability and add LLM judge tests
- Extract MAX_REPORT_LENGTH constant in llm_judge.py for maintainability
- Add comprehensive unit tests for LLMJudge class (parse_response,
calculate_weighted_score, evaluate with mocked LLM)
- Pass reportStyle prop to EvaluationDialog for accurate evaluation criteria
- Add researchQueries store map to reliably associate queries with research
- Add getResearchQuery helper to retrieve query by researchId
- Remove unused imports in test_metrics.py
* fix(eval): use resolveServiceURL for evaluate API endpoint
The evaluateReport function was using a relative URL '/api/report/evaluate'
which sent requests to the Next.js server instead of the FastAPI backend.
Changed to use resolveServiceURL() consistent with other API functions.
* fix: improve type accuracy and React hooks in evaluation components
- Fix get_word_count_target return type from Optional[Dict] to Dict since it always returns a value via default fallback
- Fix useEffect dependency issue in EvaluationDialog using useRef to prevent unwanted re-evaluations
- Add aria-label to GradeBadge for screen reader accessibility
* test: add unit tests for global connection pool (Issue #778)
- Add TestLifespanFunction class with 9 tests for lifespan management:
- PostgreSQL/MongoDB pool initialization success/failure
- Cleanup on shutdown
- Skip initialization when not configured
- Add TestGlobalConnectionPoolUsage class with 4 tests:
- Using global pools when available
- Fallback to per-request connections
- Fix missing dict_row import in app.py (bug from PR #757)
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat: add Serper search engine support
* docs: update configuration guide and env example for Serper
* test: add test case for Serper with missing API key
* feat: add enable_web_search config to disable web search (#681)
* fix: skip enforce_researcher_search validation when web search is disabled
- Return json.dumps([]) instead of empty string for consistency in background_investigation_node
- Add enable_web_search check to skip validation warning when user intentionally disabled web search
- Add warning log when researcher has no tools available
- Update tests to include new enable_web_search parameter
* fix: address Copilot review feedback
- Coordinate enforce_web_search with enable_web_search in validate_and_fix_plan
- Fix misleading comment in background_investigation_node
* docs: add warning about local RAG setup when disabling web search
* docs: add web search toggle section to configuration guide
* fix: handle greetings without triggering research workflow (#733)
* test: update tests for direct_response tool behavior
* fix: address Copilot review comments for coordinator_node - Extract locale from direct_response tool_args - Fix import sorting (ruff I001)
* fix: remove locale extraction from tool_args in direct_response
Use locale from state instead of tool_args to avoid potential side effects. The locale is already properly passed from frontend via state.
* fix: only fallback to planner when clarification is enabled
In legacy mode (BRANCH 1), no tool calls should end the workflow gracefully instead of falling back to planner. This fixes the test_coordinator_node_no_tool_calls integration test.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
