From 8bb14fa1a7bf7e6ee0631db4db9168962edbc31f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adem=20Akdo=C4=9Fan?= <53964471+ademakdogan@users.noreply.github.com> Date: Sun, 5 Apr 2026 05:19:35 +0300 Subject: [PATCH 01/12] feat(skills): add academic-paper-review, code-documentation, and newsletter-generation skills (#1861) Add three new public skills to enhance DeerFlow's content creation capabilities: - **academic-paper-review**: Structured peer-review-quality analysis of research papers following top-venue review standards (NeurIPS, ICML, ACL). Covers methodology assessment, contribution evaluation, literature positioning, and constructive feedback with a 3-phase workflow. - **code-documentation**: Professional documentation generation for software projects, including README generation, API reference docs, architecture documentation with Mermaid diagrams, and inline code documentation supporting Python, TypeScript, Go, Rust, and Java conventions. - **newsletter-generation**: Curated newsletter creation with research workflow, supporting daily digest, weekly roundup, deep-dive, and industry briefing formats. Includes audience-specific tone adaptation and multi-source content curation. All skills: - Follow the existing SKILL.md frontmatter convention (name + description) - Pass the official _validate_skill_frontmatter() validation - Use hyphen-case naming consistent with existing skills - Contain only allowed frontmatter properties - Include comprehensive examples, quality checklists, and output templates --- skills/public/academic-paper-review/SKILL.md | 289 +++++++++++++ skills/public/code-documentation/SKILL.md | 415 +++++++++++++++++++ skills/public/newsletter-generation/SKILL.md | 343 +++++++++++++++ 3 files changed, 1047 insertions(+) create mode 100644 skills/public/academic-paper-review/SKILL.md create mode 100644 skills/public/code-documentation/SKILL.md create mode 100644 skills/public/newsletter-generation/SKILL.md diff --git a/skills/public/academic-paper-review/SKILL.md b/skills/public/academic-paper-review/SKILL.md new file mode 100644 index 000000000..165321cb6 --- /dev/null +++ b/skills/public/academic-paper-review/SKILL.md @@ -0,0 +1,289 @@ +--- +name: academic-paper-review +description: Use this skill when the user requests to review, analyze, critique, or summarize academic papers, research articles, preprints, or scientific publications. Supports comprehensive structured reviews covering methodology assessment, contribution evaluation, literature positioning, and constructive feedback generation. Trigger on queries involving paper URLs, uploaded PDFs, arXiv links, or requests like "review this paper", "analyze this research", "summarize this study", or "write a peer review". +--- + +# Academic Paper Review Skill + +## Overview + +This skill produces structured, peer-review-quality analyses of academic papers and research publications. It follows established academic review standards used by top-tier venues (NeurIPS, ICML, ACL, Nature, IEEE) to provide rigorous, constructive, and balanced assessments. + +The review covers **summary, strengths, weaknesses, methodology assessment, contribution evaluation, literature positioning, and actionable recommendations** — all grounded in evidence from the paper itself. + +## Core Capabilities + +- Parse and comprehend academic papers from uploaded PDFs or fetched URLs +- Generate structured reviews following top-venue review templates +- Assess methodology rigor (experimental design, statistical validity, reproducibility) +- Evaluate novelty and significance of contributions +- Position the work within the broader research landscape via targeted literature search +- Identify limitations, gaps, and potential improvements +- Produce both detailed review and concise executive summary formats +- Support papers in any scientific domain (CS, biology, physics, social sciences, etc.) + +## When to Use This Skill + +**Always load this skill when:** + +- User provides a paper URL (arXiv, DOI, conference proceedings, journal link) +- User uploads a PDF of a research paper or preprint +- User asks to "review", "analyze", "critique", "assess", or "summarize" a research paper +- User wants to understand the strengths and weaknesses of a study +- User requests a peer-review-style evaluation of academic work +- User asks for help preparing a review for a conference or journal submission + +## Review Methodology + +### Phase 1: Paper Comprehension + +Thoroughly read and understand the paper before forming any judgments. + +#### Step 1.1: Identify Paper Metadata + +Extract and record: + +| Field | Description | +|-------|-------------| +| **Title** | Full paper title | +| **Authors** | Author list and affiliations | +| **Venue / Status** | Publication venue, preprint server, or submission status | +| **Year** | Publication or submission year | +| **Domain** | Research field and subfield | +| **Paper Type** | Empirical, theoretical, survey, position paper, systems paper, etc. | + +#### Step 1.2: Deep Reading Pass + +Read the paper systematically: + +1. **Abstract & Introduction** — Identify the claimed contributions and motivation +2. **Related Work** — Note how authors position their work relative to prior art +3. **Methodology** — Understand the proposed approach, model, or framework in detail +4. **Experiments / Results** — Examine datasets, baselines, metrics, and reported outcomes +5. **Discussion & Limitations** — Note any self-identified limitations +6. **Conclusion** — Compare concluded claims against actual evidence presented + +#### Step 1.3: Key Claims Extraction + +List the paper's main claims explicitly: + +``` +Claim 1: [Specific claim about contribution or finding] +Evidence: [What evidence supports this claim in the paper] +Strength: [Strong / Moderate / Weak] + +Claim 2: [...] +... +``` + +### Phase 2: Critical Analysis + +#### Step 2.1: Literature Context Search + +Use web search to understand the research landscape: + +``` +Search queries: +- "[paper topic] state of the art [current year]" +- "[key method name] comparison benchmark" +- "[authors] previous work [topic]" +- "[specific technique] limitations criticism" +- "survey [research area] recent advances" +``` + +Use `web_fetch` on key related papers or surveys to understand where this work fits. + +#### Step 2.2: Methodology Assessment + +Evaluate the methodology using the following framework: + +| Criterion | Questions to Ask | Rating | +|-----------|-----------------|--------| +| **Soundness** | Is the approach technically correct? Are there logical flaws? | 1-5 | +| **Novelty** | What is genuinely new vs. incremental improvement? | 1-5 | +| **Reproducibility** | Are details sufficient to reproduce? Code/data available? | 1-5 | +| **Experimental Design** | Are baselines fair? Are ablations adequate? Are datasets appropriate? | 1-5 | +| **Statistical Rigor** | Are results statistically significant? Error bars reported? Multiple runs? | 1-5 | +| **Scalability** | Does the approach scale? Are computational costs discussed? | 1-5 | + +#### Step 2.3: Contribution Significance Assessment + +Evaluate the significance level: + +| Level | Description | Criteria | +|-------|-------------|----------| +| **Landmark** | Fundamentally changes the field | New paradigm, widely applicable breakthrough | +| **Significant** | Strong contribution advancing the state of the art | Clear improvement with solid evidence | +| **Moderate** | Useful contribution with some limitations | Incremental but valid improvement | +| **Marginal** | Minimal advance over existing work | Small gains, narrow applicability | +| **Below threshold** | Does not meet publication standards | Fundamental flaws, insufficient evidence | + +#### Step 2.4: Strengths and Weaknesses Analysis + +For each strength or weakness, provide: +- **What**: Specific observation +- **Where**: Section/figure/table reference +- **Why it matters**: Impact on the paper's claims or utility + +### Phase 3: Review Synthesis + +#### Step 3.1: Assemble the Structured Review + +Produce the final review using the template below. + +## Review Output Template + +```markdown +# Paper Review: [Paper Title] + +## Paper Metadata +- **Authors**: [Author list] +- **Venue**: [Publication venue or preprint server] +- **Year**: [Year] +- **Domain**: [Research field] +- **Paper Type**: [Empirical / Theoretical / Survey / Systems / Position] + +## Executive Summary + +[2-3 paragraph summary of the paper's core contribution, approach, and main findings. +State your overall assessment upfront: what the paper does well, where it falls short, +and whether the contribution is sufficient for the claimed venue/impact level.] + +## Summary of Contributions + +1. [First claimed contribution — one sentence] +2. [Second claimed contribution — one sentence] +3. [Additional contributions if any] + +## Strengths + +### S1: [Concise strength title] +[Detailed explanation with specific references to sections, figures, or tables in the paper. +Explain WHY this is a strength and its significance.] + +### S2: [Concise strength title] +[...] + +### S3: [Concise strength title] +[...] + +## Weaknesses + +### W1: [Concise weakness title] +[Detailed explanation with specific references. Explain the impact of this weakness on +the paper's claims. Suggest how it could be addressed.] + +### W2: [Concise weakness title] +[...] + +### W3: [Concise weakness title] +[...] + +## Methodology Assessment + +| Criterion | Rating (1-5) | Assessment | +|-----------|:---:|------------| +| Soundness | X | [Brief justification] | +| Novelty | X | [Brief justification] | +| Reproducibility | X | [Brief justification] | +| Experimental Design | X | [Brief justification] | +| Statistical Rigor | X | [Brief justification] | +| Scalability | X | [Brief justification] | + +## Questions for the Authors + +1. [Specific question that would clarify a concern or ambiguity] +2. [Question about methodology choices or alternative approaches] +3. [Question about generalizability or practical applicability] + +## Minor Issues + +- [Typos, formatting issues, unclear figures, notation inconsistencies] +- [Missing references that should be cited] +- [Suggestions for improved clarity] + +## Literature Positioning + +[How does this work relate to the current state of the art? +Are key related works cited? Are comparisons fair and comprehensive? +What important related work is missing?] + +## Recommendations + +**Overall Assessment**: [Accept / Weak Accept / Borderline / Weak Reject / Reject] + +**Confidence**: [High / Medium / Low] — [Justification for confidence level] + +**Contribution Level**: [Landmark / Significant / Moderate / Marginal / Below threshold] + +### Actionable Suggestions for Improvement +1. [Specific, constructive suggestion] +2. [Specific, constructive suggestion] +3. [Specific, constructive suggestion] +``` + +## Review Principles + +### Constructive Criticism +- **Always suggest how to fix it** — Don't just point out problems; propose solutions +- **Give credit where due** — Acknowledge genuine contributions even in flawed papers +- **Be specific** — Reference exact sections, equations, figures, and tables +- **Separate minor from major** — Distinguish fatal flaws from fixable issues + +### Objectivity Standards +- ❌ "This paper is poorly written" (vague, unhelpful) +- ✅ "Section 3.2 introduces notation X without formal definition, making the proof in Theorem 1 difficult to follow. Consider adding a notation table after the problem formulation." (specific, actionable) + +### Ethical Review Practices +- Do NOT dismiss work based on author reputation or affiliation +- Evaluate the work on its own merits +- Flag potential ethical concerns (bias in datasets, dual-use implications) constructively +- Maintain confidentiality of unpublished work + +## Adaptation by Paper Type + +| Paper Type | Focus Areas | +|------------|-------------| +| **Empirical** | Experimental design, baselines, statistical significance, ablations, reproducibility | +| **Theoretical** | Proof correctness, assumption reasonableness, tightness of bounds, connection to practice | +| **Survey** | Comprehensiveness, taxonomy quality, coverage of recent work, synthesis insights | +| **Systems** | Architecture decisions, scalability evidence, real-world deployment, engineering contributions | +| **Position** | Argument coherence, evidence for claims, impact potential, fairness of characterizations | + +## Common Pitfalls to Avoid + +- ❌ Reviewing the paper you wish was written instead of the paper that was submitted +- ❌ Demanding additional experiments that are unreasonable in scope +- ❌ Penalizing the paper for not solving a different problem +- ❌ Being overly influenced by writing quality versus technical contribution +- ❌ Treating absence of comparison to your own work as a weakness +- ❌ Providing only a summary without critical analysis + +## Quality Checklist + +Before finalizing the review, verify: + +- [ ] Paper was read completely (not just abstract and introduction) +- [ ] All major claims are identified and evaluated against evidence +- [ ] At least 3 strengths and 3 weaknesses are provided with specific references +- [ ] The methodology assessment table is complete with ratings and justifications +- [ ] Questions for authors target genuine ambiguities, not rhetorical critiques +- [ ] Literature search was conducted to contextualize the contribution +- [ ] Recommendations are actionable and constructive +- [ ] The overall assessment is consistent with the identified strengths and weaknesses +- [ ] The review tone is professional and respectful +- [ ] Minor issues are separated from major concerns + +## Output Format + +- Output the complete review in **Markdown** format +- Save the review to `/mnt/user-data/outputs/review-{paper-topic}.md` when working in sandbox +- Present the review to the user using the `present_files` tool + +## Notes + +- This skill complements the `deep-research` skill — load both when the user wants the paper reviewed in the context of the broader field +- For papers behind paywalls, work with whatever content is accessible (abstract, publicly available versions, preprint mirrors) +- Adapt the review depth to the user's needs: a brief assessment for quick triage versus a full review for submission preparation +- When reviewing multiple papers comparatively, maintain consistent criteria across all reviews +- Always disclose limitations of your review (e.g., "I could not verify the proofs in Appendix B in detail") diff --git a/skills/public/code-documentation/SKILL.md b/skills/public/code-documentation/SKILL.md new file mode 100644 index 000000000..8a2e2c47b --- /dev/null +++ b/skills/public/code-documentation/SKILL.md @@ -0,0 +1,415 @@ +--- +name: code-documentation +description: Use this skill when the user requests to generate, create, or improve documentation for code, APIs, libraries, repositories, or software projects. Supports README generation, API reference documentation, inline code comments, architecture documentation, changelog generation, and developer guides. Trigger on requests like "document this code", "create a README", "generate API docs", "write developer guide", or when analyzing codebases for documentation purposes. +--- + +# Code Documentation Skill + +## Overview + +This skill generates professional, comprehensive documentation for software projects, codebases, libraries, and APIs. It follows industry best practices from projects like React, Django, Stripe, and Kubernetes to produce documentation that is accurate, well-structured, and useful for both new contributors and experienced developers. + +The output ranges from single-file READMEs to multi-document developer guides, always matched to the project's complexity and the user's needs. + +## Core Capabilities + +- Generate comprehensive README.md files with badges, installation, usage, and API reference +- Create API reference documentation from source code analysis +- Produce architecture and design documentation with diagrams +- Write developer onboarding and contribution guides +- Generate changelogs from commit history or release notes +- Create inline code documentation following language-specific conventions +- Support JSDoc, docstrings, GoDoc, Javadoc, and Rustdoc formats +- Adapt documentation style to the project's language and ecosystem + +## When to Use This Skill + +**Always load this skill when:** + +- User asks to "document", "create docs", or "write documentation" for any code +- User requests a README, API reference, or developer guide +- User shares a codebase or repository and wants documentation generated +- User asks to improve or update existing documentation +- User needs architecture documentation, including diagrams +- User requests a changelog or migration guide + +## Documentation Workflow + +### Phase 1: Codebase Analysis + +Before writing any documentation, thoroughly understand the codebase. + +#### Step 1.1: Project Discovery + +Identify the project fundamentals: + +| Field | How to Determine | +|-------|-----------------| +| **Language(s)** | Check file extensions, `package.json`, `pyproject.toml`, `go.mod`, `Cargo.toml`, etc. | +| **Framework** | Look at dependencies for known frameworks (React, Django, Express, Spring, etc.) | +| **Build System** | Check for `Makefile`, `CMakeLists.txt`, `webpack.config.js`, `build.gradle`, etc. | +| **Package Manager** | npm/yarn/pnpm, pip/uv/poetry, cargo, go modules, etc. | +| **Project Structure** | Map out the directory tree to understand the architecture | +| **Entry Points** | Find main files, CLI entry points, exported modules | +| **Existing Docs** | Check for existing README, docs/, wiki, or inline documentation | + +#### Step 1.2: Code Structure Analysis + +Use sandbox tools to explore the codebase: + +```bash +# Get directory structure +ls /mnt/user-data/uploads/project-dir/ + +# Read key files +read_file /mnt/user-data/uploads/project-dir/package.json +read_file /mnt/user-data/uploads/project-dir/pyproject.toml + +# Search for public API surfaces +grep -r "export " /mnt/user-data/uploads/project-dir/src/ +grep -r "def " /mnt/user-data/uploads/project-dir/src/ --include="*.py" +grep -r "func " /mnt/user-data/uploads/project-dir/ --include="*.go" +``` + +#### Step 1.3: Identify Documentation Scope + +Based on analysis, determine what documentation to produce: + +| Project Size | Recommended Documentation | +|-------------|--------------------------| +| **Single file / script** | Inline comments + usage header | +| **Small library** | README with API reference | +| **Medium project** | README + API docs + examples | +| **Large project** | README + Architecture + API + Contributing + Changelog | + +### Phase 2: Documentation Generation + +#### Step 2.1: README Generation + +Every project needs a README. Follow this structure: + +```markdown +# Project Name + +[One-line project description — what it does and why it matters] + +[![Badge](link)](#) [![Badge](link)](#) + +## Features + +- [Key feature 1 — brief description] +- [Key feature 2 — brief description] +- [Key feature 3 — brief description] + +## Quick Start + +### Prerequisites + +- [Prerequisite 1 with version requirement] +- [Prerequisite 2 with version requirement] + +### Installation + +[Installation commands with copy-paste-ready code blocks] + +### Basic Usage + +[Minimal working example that demonstrates core functionality] + +## Documentation + +- [Link to full API reference if separate] +- [Link to architecture docs if separate] +- [Link to examples directory if applicable] + +## API Reference + +[Inline API reference for smaller projects OR link to generated docs] + +## Configuration + +[Environment variables, config files, or runtime options] + +## Examples + +[2-3 practical examples covering common use cases] + +## Development + +### Setup + +[How to set up a development environment] + +### Testing + +[How to run tests] + +### Building + +[How to build the project] + +## Contributing + +[Contribution guidelines or link to CONTRIBUTING.md] + +## License + +[License information] +``` + +#### Step 2.2: API Reference Generation + +For each public API surface, document: + +**Function / Method Documentation**: + +```markdown +### `functionName(param1, param2, options?)` + +Brief description of what this function does. + +**Parameters:** + +| Parameter | Type | Required | Default | Description | +|-----------|------|----------|---------|-------------| +| `param1` | `string` | Yes | — | Description of param1 | +| `param2` | `number` | Yes | — | Description of param2 | +| `options` | `Object` | No | `{}` | Configuration options | +| `options.timeout` | `number` | No | `5000` | Timeout in milliseconds | + +**Returns:** `Promise` — Description of return value + +**Throws:** +- `ValidationError` — When param1 is empty +- `TimeoutError` — When the operation exceeds the timeout + +**Example:** + +\`\`\`javascript +const result = await functionName("hello", 42, { timeout: 10000 }); +console.log(result.data); +\`\`\` +``` + +**Class Documentation**: + +```markdown +### `ClassName` + +Brief description of the class and its purpose. + +**Constructor:** + +\`\`\`javascript +new ClassName(config) +\`\`\` + +| Parameter | Type | Description | +|-----------|------|-------------| +| `config.option1` | `string` | Description | +| `config.option2` | `boolean` | Description | + +**Methods:** + +- [`method1()`](#method1) — Brief description +- [`method2(param)`](#method2) — Brief description + +**Properties:** + +| Property | Type | Description | +|----------|------|-------------| +| `property1` | `string` | Description | +| `property2` | `number` | Read-only. Description | +``` + +#### Step 2.3: Architecture Documentation + +For medium-to-large projects, include architecture documentation: + +```markdown +# Architecture Overview + +## System Diagram + +[Include a Mermaid diagram showing the high-level architecture] + +\`\`\`mermaid +graph TD + A[Client] --> B[API Gateway] + B --> C[Service A] + B --> D[Service B] + C --> E[(Database)] + D --> E +\`\`\` + +## Component Overview + +### Component Name +- **Purpose**: What this component does +- **Location**: `src/components/name/` +- **Dependencies**: What it depends on +- **Public API**: Key exports or interfaces + +## Data Flow + +[Describe how data flows through the system for key operations] + +## Design Decisions + +### Decision Title +- **Context**: What situation led to this decision +- **Decision**: What was decided +- **Rationale**: Why this approach was chosen +- **Trade-offs**: What was sacrificed +``` + +#### Step 2.4: Inline Code Documentation + +Generate language-appropriate inline documentation: + +**Python (Docstrings — Google style)**: +```python +def process_data(input_path: str, options: dict | None = None) -> ProcessResult: + """Process data from the given file path. + + Reads the input file, applies transformations based on the provided + options, and returns a structured result object. + + Args: + input_path: Absolute path to the input data file. + Supports CSV, JSON, and Parquet formats. + options: Optional configuration dictionary. + - "validate" (bool): Enable input validation. Defaults to True. + - "format" (str): Output format ("json" or "csv"). Defaults to "json". + + Returns: + A ProcessResult containing the transformed data and metadata. + + Raises: + FileNotFoundError: If input_path does not exist. + ValidationError: If validation is enabled and data is malformed. + + Example: + >>> result = process_data("/data/input.csv", {"validate": True}) + >>> print(result.row_count) + 1500 + """ +``` + +**TypeScript (JSDoc / TSDoc)**: +```typescript +/** + * Fetches user data from the API and transforms it for display. + * + * @param userId - The unique identifier of the user + * @param options - Configuration options for the fetch operation + * @param options.includeProfile - Whether to include the full profile. Defaults to `false`. + * @param options.cache - Cache duration in seconds. Set to `0` to disable. + * @returns The transformed user data ready for rendering + * @throws {NotFoundError} When the user ID does not exist + * @throws {NetworkError} When the API is unreachable + * + * @example + * ```ts + * const user = await fetchUser("usr_123", { includeProfile: true }); + * console.log(user.displayName); + * ``` + */ +``` + +**Go (GoDoc)**: +```go +// ProcessData reads the input file at the given path, applies the specified +// transformations, and returns the processed result. +// +// The input path must be an absolute path to a CSV or JSON file. +// If options is nil, default options are used. +// +// ProcessData returns an error if the file does not exist or cannot be parsed. +func ProcessData(inputPath string, options *ProcessOptions) (*Result, error) { +``` + +### Phase 3: Quality Assurance + +#### Step 3.1: Documentation Completeness Check + +Verify the documentation covers: + +- [ ] **What it is** — Clear project description that a newcomer can understand +- [ ] **Why it exists** — Problem it solves and value proposition +- [ ] **How to install** — Copy-paste-ready installation commands +- [ ] **How to use** — At least one minimal working example +- [ ] **API surface** — All public functions, classes, and types documented +- [ ] **Configuration** — All environment variables, config files, and options +- [ ] **Error handling** — Common errors and how to resolve them +- [ ] **Contributing** — How to set up dev environment and submit changes + +#### Step 3.2: Quality Standards + +| Standard | Check | +|----------|-------| +| **Accuracy** | Every code example must actually work with the described API | +| **Completeness** | No public API surface left undocumented | +| **Consistency** | Same formatting and structure throughout | +| **Freshness** | Documentation matches the current code, not an older version | +| **Accessibility** | No jargon without explanation, acronyms defined on first use | +| **Examples** | Every complex concept has at least one practical example | + +#### Step 3.3: Cross-reference Validation + +Ensure: +- All mentioned file paths exist in the project +- All referenced functions and classes exist in the code +- All code examples use the correct function signatures +- Version numbers match the project's actual version +- All links (internal and external) are valid + +## Documentation Style Guide + +### Writing Principles + +1. **Lead with the "why"** — Before explaining how something works, explain why it exists +2. **Progressive disclosure** — Start simple, add complexity gradually +3. **Show, don't tell** — Prefer code examples over lengthy explanations +4. **Active voice** — "The function returns X" not "X is returned by the function" +5. **Present tense** — "The server starts on port 8080" not "The server will start on port 8080" +6. **Second person** — "You can configure..." not "Users can configure..." + +### Formatting Rules + +- Use ATX-style headers (`#`, `##`, `###`) +- Use fenced code blocks with language specification (` ```python `, ` ```bash `) +- Use tables for structured information (parameters, options, configuration) +- Use admonitions for important notes, warnings, and tips +- Keep line length readable (wrap prose at ~80-100 characters in source) +- Use `code formatting` for function names, file paths, variable names, and CLI commands + +### Language-Specific Conventions + +| Language | Doc Format | Style Guide | +|----------|-----------|-------------| +| Python | Google-style docstrings | PEP 257 | +| TypeScript/JavaScript | TSDoc / JSDoc | TypeDoc conventions | +| Go | GoDoc comments | Effective Go | +| Rust | Rustdoc (`///`) | Rust API Guidelines | +| Java | Javadoc | Oracle Javadoc Guide | +| C/C++ | Doxygen | Doxygen manual | + +## Output Handling + +After generation: + +- Save documentation files to `/mnt/user-data/outputs/` +- For multi-file documentation, maintain the project directory structure +- Present generated files to the user using the `present_files` tool +- Offer to iterate on specific sections or adjust the level of detail +- Suggest additional documentation that might be valuable + +## Notes + +- Always analyze the actual code before writing documentation — never guess at API signatures or behavior +- When existing documentation exists, preserve its structure unless the user explicitly asks for a rewrite +- For large codebases, prioritize documenting the public API surface and key abstractions first +- Documentation should be written in the same language as the project's existing docs; default to English if none exist +- When generating changelogs, use the [Keep a Changelog](https://keepachangelog.com/) format +- This skill works well in combination with the `deep-research` skill for documenting third-party integrations or dependencies diff --git a/skills/public/newsletter-generation/SKILL.md b/skills/public/newsletter-generation/SKILL.md new file mode 100644 index 000000000..0f0221e50 --- /dev/null +++ b/skills/public/newsletter-generation/SKILL.md @@ -0,0 +1,343 @@ +--- +name: newsletter-generation +description: Use this skill when the user requests to generate, create, write, or draft a newsletter, email digest, weekly roundup, industry briefing, or curated content summary. Supports topic-based research, content curation from multiple sources, and professional formatting for email or web distribution. Trigger on requests like "create a newsletter about X", "write a weekly digest", "generate a tech roundup", or "curate news about Y". +--- + +# Newsletter Generation Skill + +## Overview + +This skill generates professional, well-researched newsletters that combine curated content from multiple sources with original analysis and commentary. It follows modern newsletter best practices from publications like Morning Brew, The Hustle, TLDR, and Benedict Evans to produce content that is informative, engaging, and actionable. + +The output is a complete, ready-to-publish newsletter in Markdown format, suitable for email distribution platforms, web publishing, or conversion to HTML. + +## Core Capabilities + +- Research and curate content from multiple web sources on specified topics +- Generate topic-focused or multi-topic newsletters with consistent voice +- Write engaging headlines, summaries, and original commentary +- Structure content for optimal readability and scanning +- Support multiple newsletter formats (daily digest, weekly roundup, deep-dive, industry briefing) +- Include relevant links, sources, and attributions +- Adapt tone and style to target audience (technical, executive, general) +- Generate recurring newsletter series with consistent branding and structure + +## When to Use This Skill + +**Always load this skill when:** + +- User asks to generate a newsletter, email digest, or content roundup +- User requests a curated summary of news or developments on a topic +- User wants to create a recurring newsletter format +- User asks to compile recent developments in a field into a briefing +- User needs a formatted email-ready content piece with multiple curated items +- User asks for a "weekly roundup", "monthly digest", or "morning briefing" + +## Newsletter Workflow + +### Phase 1: Planning + +#### Step 1.1: Understand Newsletter Requirements + +Identify the key parameters: + +| Parameter | Description | Default | +|-----------|-------------|---------| +| **Topic(s)** | Primary subject area(s) to cover | Required | +| **Format** | Daily digest, weekly roundup, deep-dive, or industry briefing | Weekly roundup | +| **Target Audience** | Technical, executive, general, or niche community | General | +| **Tone** | Professional, conversational, witty, or analytical | Conversational-professional | +| **Length** | Short (5-min read), medium (10-min), long (15-min+) | Medium | +| **Sections** | Number and type of content sections | 4-6 sections | +| **Frequency Context** | One-time or part of a recurring series | One-time | + +#### Step 1.2: Define Newsletter Structure + +Based on the format, select the appropriate structure: + +**Daily Digest Structure**: +``` +1. Top Story (1 item, detailed) +2. Quick Hits (3-5 items, brief) +3. One Stat / Quote of the Day +4. What to Watch +``` + +**Weekly Roundup Structure**: +``` +1. Editor's Note / Intro +2. Top Stories (2-3 items, detailed) +3. Trends & Analysis (1-2 items, original commentary) +4. Quick Bites (4-6 items, brief summaries) +5. Tools & Resources (2-3 items) +6. One More Thing / Closing +``` + +**Deep-Dive Structure**: +``` +1. Introduction & Context +2. Background / Why It Matters +3. Key Developments (detailed analysis) +4. Expert Perspectives +5. What's Next / Implications +6. Further Reading +``` + +**Industry Briefing Structure**: +``` +1. Executive Summary +2. Market Developments +3. Company News & Moves +4. Product & Technology Updates +5. Regulatory & Policy Changes +6. Data & Metrics +7. Outlook +``` + +### Phase 2: Research & Curation + +#### Step 2.1: Multi-Source Research + +Conduct thorough research using web search. **The quality of the newsletter depends directly on the quality and recency of research.** + +**Search Strategy**: + +``` +# Current news and developments +"[topic] news [current month] [current year]" +"[topic] latest developments" +"[topic] announcement this week" + +# Trends and analysis +"[topic] trends [current year]" +"[topic] analysis expert opinion" +"[topic] industry report" + +# Data and statistics +"[topic] statistics [current year]" +"[topic] market data latest" +"[topic] growth metrics" + +# Tools and resources +"[topic] new tools [current year]" +"[topic] open source release" +"best [topic] resources [current year]" +``` + +> **IMPORTANT**: Always check `` to ensure search queries use the correct temporal context. Never use hardcoded years. + +#### Step 2.2: Source Evaluation and Selection + +Evaluate each source and curate the best content: + +| Criterion | Priority | +|-----------|----------| +| **Recency** | Prefer content from the last 7-30 days | +| **Authority** | Prioritize primary sources, official announcements, established publications | +| **Uniqueness** | Select stories that offer fresh perspective or are underreported | +| **Relevance** | Every item must clearly connect to the newsletter's stated topic(s) | +| **Actionability** | Prefer content readers can act on (tools, insights, strategies) | +| **Diversity** | Mix of news, analysis, data, and practical resources | + +#### Step 2.3: Deep Content Extraction + +For key stories, use `web_fetch` to read full articles and extract: + +1. **Core facts** — What happened, who is involved, when +2. **Context** — Why this matters, background information +3. **Data points** — Specific numbers, metrics, or statistics +4. **Quotes** — Relevant expert quotes or official statements +5. **Implications** — What this means for the reader + +### Phase 3: Writing + +#### Step 3.1: Newsletter Header + +Every newsletter starts with a consistent header: + +```markdown +# [Newsletter Name] + +*[Tagline or description] — [Date]* + +--- + +[Optional: One-sentence preview of what's inside] +``` + +#### Step 3.2: Section Writing Guidelines + +**Top Stories / Featured Items**: +- **Headline**: Compelling, clear, benefit-oriented (not clickbait) +- **Hook**: Opening sentence that makes the reader care (1-2 sentences) +- **Body**: Key facts and context (2-4 paragraphs) +- **Why it matters**: Connect to the reader's world (1 paragraph) +- **Source link**: Always attribute and link to the original source + +**Quick Bites / Brief Items**: +- **Format**: Bold headline + 2-3 sentence summary + source link +- **Focus**: One key takeaway per item +- **Efficiency**: Readers should get the essential insight without clicking through + +**Analysis / Commentary Sections**: +- **Voice**: The newsletter's unique perspective on trends or developments +- **Structure**: Observation → Context → Implication → (Optional) Actionable takeaway +- **Evidence**: Every claim backed by data or sourced information + +#### Step 3.3: Writing Standards + +| Principle | Implementation | +|-----------|---------------| +| **Scannable** | Use headers, bold text, bullet points, and short paragraphs | +| **Engaging** | Lead with the most interesting angle, not chronological order | +| **Concise** | Every sentence earns its place — cut filler ruthlessly | +| **Accurate** | Every fact is sourced, every number is verified | +| **Attributive** | Always credit original sources with inline links | +| **Human** | Write like a knowledgeable friend, not a press release | + +**Tone Calibration by Audience**: + +| Audience | Tone | Example | +|----------|------|---------| +| **Technical** | Precise, no jargon explanations, assumed expertise | "The new API supports gRPC streaming with backpressure handling via flow control windows." | +| **Executive** | Impact-focused, bottom-line, strategic | "This acquisition gives Company X a 40% market share in the enterprise segment, directly threatening Incumbent Y's pricing power." | +| **General** | Accessible, analogies, explains concepts | "Think of it like a universal translator for data — it lets any app talk to any database without learning a new language." | + +### Phase 4: Assembly & Polish + +#### Step 4.1: Assemble the Newsletter + +Combine all sections into the final document following the chosen structure template. + +#### Step 4.2: Footer + +Every newsletter ends with: + +```markdown +--- + +*[Newsletter Name] is [description of what it is].* +*[How to subscribe/share/give feedback]* + +*Sources: All links are provided inline. This newsletter curates and summarizes +publicly available information with original commentary.* +``` + +#### Step 4.3: Quality Checklist + +Before finalizing, verify: + +- [ ] **Every factual claim has a source link** — No unsourced assertions +- [ ] **All links are functional** — Verified URLs from search results +- [ ] **Date references use the actual current date** — No hardcoded or assumed dates +- [ ] **Content is current** — All major items are from within the expected timeframe +- [ ] **No duplicate stories** — Each item appears only once +- [ ] **Consistent formatting** — Headers, bullets, links use the same style throughout +- [ ] **Balanced coverage** — Not dominated by a single source or perspective +- [ ] **Appropriate length** — Matches the specified length target +- [ ] **Engaging opening** — The first 2 sentences make the reader want to continue +- [ ] **Clear closing** — The newsletter ends with a memorable or actionable note +- [ ] **Proofread** — No typos, broken formatting, or incomplete sentences + +## Newsletter Output Template + +```markdown +# [Newsletter Name] + +*[Tagline] — [Full date, e.g., April 4, 2026]* + +--- + +[Preview sentence: "This week: [topic 1], [topic 2], and [topic 3]."] + +## 🔥 Top Stories + +### [Headline 1] + +[Hook — why this matters in 1-2 sentences.] + +[Body — 2-4 paragraphs covering key facts, context, and implications.] + +**Why it matters:** [1 paragraph connecting to reader's interests or industry impact.] + +📎 [Source: Publication Name](URL) + +### [Headline 2] + +[Same structure as above] + +## 📊 Trends & Analysis + +### [Trend Title] + +[Original commentary on an emerging trend, backed by data from research.] + +[Key data points presented clearly — consider inline stats or a brief comparison.] + +**The bottom line:** [One-sentence takeaway.] + +## ⚡ Quick Bites + +- **[Headline]** — [2-3 sentence summary with key takeaway.] [Source](URL) +- **[Headline]** — [2-3 sentence summary.] [Source](URL) +- **[Headline]** — [2-3 sentence summary.] [Source](URL) +- **[Headline]** — [2-3 sentence summary.] [Source](URL) + +## 🛠️ Tools & Resources + +- **[Tool/Resource Name]** — [What it does and why it's useful.] [Link](URL) +- **[Tool/Resource Name]** — [Description.] [Link](URL) + +## 💬 One More Thing + +[Closing thought, insightful quote, or forward-looking statement.] + +--- + +*[Newsletter Name] curates the most important [topic] news and analysis.* +*Found this useful? Share it with a colleague.* + +*All sources are linked inline. Views and commentary are original.* +``` + +## Adaptation Examples + +### Technology Newsletter +- Emoji usage: ✅ Moderate (section headers) +- Sections: Top Stories, Deep Dive, Quick Bites, Open Source Spotlight, Dev Tools +- Tone: Technical-conversational + +### Business/Finance Newsletter +- Emoji usage: ❌ Minimal to none +- Sections: Market Overview, Deal Flow, Company News, Data Corner, Outlook +- Tone: Professional-analytical + +### Industry-Specific Newsletter +- Emoji usage: Moderate +- Sections: Regulatory Updates, Market Data, Innovation Watch, People Moves, Events +- Tone: Expert-authoritative + +### Creative/Marketing Newsletter +- Emoji usage: ✅ Liberal +- Sections: Campaign Spotlight, Trend Watch, Viral This Week, Tools We Love, Inspiration +- Tone: Enthusiastic-professional + +## Output Handling + +After generation: + +- Save the newsletter to `/mnt/user-data/outputs/newsletter-{topic}-{date}.md` +- Present the newsletter to the user using the `present_files` tool +- Offer to adjust sections, tone, length, or focus areas +- If the user wants HTML output, note that the Markdown can be converted using standard tools + +## Notes + +- This skill works best in combination with the `deep-research` skill for comprehensive topic coverage — load both for newsletters requiring deep analysis +- Always use `` for temporal context in searches and date references in the newsletter +- For recurring newsletters, suggest maintaining a consistent structure so readers develop expectations +- When curating, quality beats quantity — 5 excellent items beat 15 mediocre ones +- Attribute all content properly — newsletters build trust through transparent sourcing +- Avoid summarizing paywalled content that the reader cannot access +- If the user provides specific URLs or articles to include, incorporate them alongside your curated findings +- The newsletter should provide enough value in the summaries that readers benefit even without clicking through to every link From 5f8dac66e6e72570a5526a04d929b59c84d1b128 Mon Sep 17 00:00:00 2001 From: yangzheli <43645580+yangzheli@users.noreply.github.com> Date: Sun, 5 Apr 2026 10:22:14 +0800 Subject: [PATCH 02/12] chore(deps): update uv.lock (#1848) Co-authored-by: Willem Jiang --- backend/uv.lock | 74 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) diff --git a/backend/uv.lock b/backend/uv.lock index 7a40d5656..45731fb04 100644 --- a/backend/uv.lock +++ b/backend/uv.lock @@ -747,6 +747,11 @@ dependencies = [ { name = "tiktoken" }, ] +[package.optional-dependencies] +pymupdf = [ + { name = "pymupdf4llm" }, +] + [package.metadata] requires-dist = [ { name = "agent-client-protocol", specifier = ">=0.4.0" }, @@ -773,11 +778,13 @@ requires-dist = [ { name = "markdownify", specifier = ">=1.2.2" }, { name = "markitdown", extras = ["all", "xlsx"], specifier = ">=0.0.1a2" }, { name = "pydantic", specifier = ">=2.12.5" }, + { name = "pymupdf4llm", marker = "extra == 'pymupdf'", specifier = ">=0.0.17" }, { name = "pyyaml", specifier = ">=6.0.3" }, { name = "readabilipy", specifier = ">=0.3.0" }, { name = "tavily-python", specifier = ">=0.7.17" }, { name = "tiktoken", specifier = ">=0.8.0" }, ] +provides-extras = ["pymupdf"] [[package]] name = "defusedxml" @@ -2149,6 +2156,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/a0/c4/c2971a3ba4c6103a3d10c4b0f24f461ddc027f0f09763220cf35ca1401b3/nest_asyncio-1.6.0-py3-none-any.whl", hash = "sha256:87af6efd6b5e897c81050477ef65c62e2b2f35d51703cae01aff2905b1852e1c", size = 5195, upload-time = "2024-01-21T14:25:17.223Z" }, ] +[[package]] +name = "networkx" +version = "3.6.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/6a/51/63fe664f3908c97be9d2e4f1158eb633317598cfa6e1fc14af5383f17512/networkx-3.6.1.tar.gz", hash = "sha256:26b7c357accc0c8cde558ad486283728b65b6a95d85ee1cd66bafab4c8168509", size = 2517025, upload-time = "2025-12-08T17:02:39.908Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/9e/c9/b2622292ea83fbb4ec318f5b9ab867d0a28ab43c5717bb85b0a5f6b3b0a4/networkx-3.6.1-py3-none-any.whl", hash = "sha256:d47fbf302e7d9cbbb9e2555a0d267983d2aa476bac30e90dfbe5669bd57f3762", size = 2068504, upload-time = "2025-12-08T17:02:38.159Z" }, +] + [[package]] name = "numpy" version = "2.4.1" @@ -2943,6 +2959,55 @@ crypto = [ { name = "cryptography" }, ] +[[package]] +name = "pymupdf" +version = "1.27.2.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f1/32/f6b645c51d79a188a4844140c5dabca7b487ad56c4be69c4bc782d0d11a9/pymupdf-1.27.2.2.tar.gz", hash = "sha256:ea8fdc3ab6671ca98f629d5ec3032d662c8cf1796b146996b7ad306ac7ed3335", size = 85354380, upload-time = "2026-03-20T09:47:58.386Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/90/88/d01992a50165e22dec057a1129826846c547feb4ba07f42720ac030ce438/pymupdf-1.27.2.2-cp310-abi3-macosx_10_9_x86_64.whl", hash = "sha256:800f43e60a6f01f644343c2213b8613db02eaf4f4ba235b417b3351fa99e01c0", size = 23987563, upload-time = "2026-03-19T12:35:42.989Z" }, + { url = "https://files.pythonhosted.org/packages/6d/0e/9f526bc1d49d8082eff0d1547a69d541a0c5a052e71da625559efaba46a6/pymupdf-1.27.2.2-cp310-abi3-macosx_11_0_arm64.whl", hash = "sha256:8e2e4299ef1ac0c9dff9be096cbd22783699673abecfa7c3f73173ae06421d73", size = 23263089, upload-time = "2026-03-20T09:44:16.982Z" }, + { url = "https://files.pythonhosted.org/packages/42/be/984f0d6343935b5dd30afaed6be04fc753146bf55709e63ef28bf9ef7497/pymupdf-1.27.2.2-cp310-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:c5e3d54922db1c7da844f1208ac1db05704770988752311f81dd36694ae0a07b", size = 24318817, upload-time = "2026-03-20T09:44:33.209Z" }, + { url = "https://files.pythonhosted.org/packages/22/8e/85e9d9f11dbf34036eb1df283805ef6b885f2005a56d6533bb58ab0b8a11/pymupdf-1.27.2.2-cp310-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:892698c9768457eb0991c102c96a856c0a7062539371df5e6bee0816f3ef498e", size = 24948135, upload-time = "2026-03-20T09:44:51.012Z" }, + { url = "https://files.pythonhosted.org/packages/db/e6/386edb017e5b93f1ab0bf6653ae32f3dd8dfc834ed770212e10ca62f4af9/pymupdf-1.27.2.2-cp310-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:8b4bbfa6ef347fade678771a93f6364971c51a2cdc44cd2400dc4eeed1ddb4e6", size = 25169585, upload-time = "2026-03-20T09:45:05.393Z" }, + { url = "https://files.pythonhosted.org/packages/ba/fd/f1ebe24fcd31aaea8b85b3a7ac4c3fc96e20388be5466ace27c9a3c546d9/pymupdf-1.27.2.2-cp310-abi3-win32.whl", hash = "sha256:0b8e924433b7e0bd46be820899300259235997d5a747638471fb2762baa8ee30", size = 18008861, upload-time = "2026-03-20T09:45:21.353Z" }, + { url = "https://files.pythonhosted.org/packages/a8/b6/2a9a8556000199bbf80a5915dcd15d550d1e5288894316445c54726aaf53/pymupdf-1.27.2.2-cp310-abi3-win_amd64.whl", hash = "sha256:09bb53f9486ccb5297030cbc2dbdae845ba1c3c5126e96eb2d16c4f118de0b5b", size = 19238032, upload-time = "2026-03-20T09:45:37.941Z" }, + { url = "https://files.pythonhosted.org/packages/c2/c6/e3e11c42f09b9c34ec332c0f37b817671b59ef4001895b854f0494092105/pymupdf-1.27.2.2-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:6cebfbbdfd219ebdebf4d8e3914624b2e3d3a844c43f4f76935822dd9b13cc12", size = 24985299, upload-time = "2026-03-20T09:45:53.26Z" }, +] + +[[package]] +name = "pymupdf-layout" +version = "1.27.2.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "networkx" }, + { name = "numpy" }, + { name = "onnxruntime" }, + { name = "pymupdf" }, + { name = "pyyaml" }, +] +wheels = [ + { url = "https://files.pythonhosted.org/packages/65/dd/4a9769b17661c1ee1b5bdeac28c832c9c7cc1ef425eb2088b5b5bd982bcc/pymupdf_layout-1.27.2.2-cp310-abi3-macosx_10_9_x86_64.whl", hash = "sha256:7b8f0d94d5675802c67e4af321214dcfce2de3d963926459dc6fc138607366cd", size = 15799842, upload-time = "2026-03-20T09:46:04.194Z" }, + { url = "https://files.pythonhosted.org/packages/ce/14/3ed13138449a002ab6957789019da5951fc8ba07ab8f1faf27a14c274717/pymupdf_layout-1.27.2.2-cp310-abi3-macosx_11_0_arm64.whl", hash = "sha256:bef82a3ff5c05212c806333153cece2b9d972eed173d2352f0c514bb3f1faf54", size = 15795217, upload-time = "2026-03-20T09:46:14.142Z" }, + { url = "https://files.pythonhosted.org/packages/f7/20/487a2b1422999113ecc8b117cf50e72915992d0a7ef247164989396cf8db/pymupdf_layout-1.27.2.2-cp310-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:d610359e1eb8013124531431f3b8c77818070e7869500b92c9b25bd78ea7ef7f", size = 15805238, upload-time = "2026-03-20T09:46:23.676Z" }, + { url = "https://files.pythonhosted.org/packages/02/45/35c67a1b1956618f69674b9823cc78e96787de37fe22a2b217581a1770a9/pymupdf_layout-1.27.2.2-cp310-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:df503eab9c28cfaadb847970f39093958e7a2ebf79fc47426dbd91b9f9064d6c", size = 15806267, upload-time = "2026-03-20T09:46:33.089Z" }, + { url = "https://files.pythonhosted.org/packages/82/56/97fad0cd00869e934f7a130f251b21e3534ec0fcffaa3459286fbf3daf32/pymupdf_layout-1.27.2.2-cp310-abi3-win_amd64.whl", hash = "sha256:efc66387833f085b9e9a77089c748c88c4c96485772d7dfe0139eaa6efc2f444", size = 15809705, upload-time = "2026-03-20T09:46:43.009Z" }, +] + +[[package]] +name = "pymupdf4llm" +version = "1.27.2.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pymupdf" }, + { name = "pymupdf-layout" }, + { name = "tabulate" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/f0/e7/8b97bf223ea2fd72efd862af3210ae3aa2fb15b39b55767de9e0a2fd0985/pymupdf4llm-1.27.2.2.tar.gz", hash = "sha256:f95e113d434958f8c63393c836fe965ad398d1fc07e7807c0a627c9ec1946e9f", size = 72877, upload-time = "2026-03-20T09:48:01.485Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/01/fc/a4977b84f9a7e70aac4c9beed55d4693b985cef89fab7d49c896335bf158/pymupdf4llm-1.27.2.2-py3-none-any.whl", hash = "sha256:ec3bbceed21c6f86289155f29c557aa54ae1c8282c4a45d6de984f16fb4c90cb", size = 84294, upload-time = "2026-03-20T09:45:55.365Z" }, +] + [[package]] name = "pypdfium2" version = "5.3.0" @@ -3542,6 +3607,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/a2/09/77d55d46fd61b4a135c444fc97158ef34a095e5681d0a6c10b75bf356191/sympy-1.14.0-py3-none-any.whl", hash = "sha256:e091cc3e99d2141a0ba2847328f5479b05d94a6635cb96148ccb3f34671bd8f5", size = 6299353, upload-time = "2025-04-27T18:04:59.103Z" }, ] +[[package]] +name = "tabulate" +version = "0.10.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/46/58/8c37dea7bbf769b20d58e7ace7e5edfe65b849442b00ffcdd56be88697c6/tabulate-0.10.0.tar.gz", hash = "sha256:e2cfde8f79420f6deeffdeda9aaec3b6bc5abce947655d17ac662b126e48a60d", size = 91754, upload-time = "2026-03-04T18:55:34.402Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/99/55/db07de81b5c630da5cbf5c7df646580ca26dfaefa593667fc6f2fe016d2e/tabulate-0.10.0-py3-none-any.whl", hash = "sha256:f0b0622e567335c8fabaaa659f1b33bcb6ddfe2e496071b743aa113f8774f2d3", size = 39814, upload-time = "2026-03-04T18:55:31.284Z" }, +] + [[package]] name = "tavily-python" version = "0.7.17" From a283d4a02d701ef9dc514727738646210fd82992 Mon Sep 17 00:00:00 2001 From: Octopus Date: Sat, 4 Apr 2026 21:49:58 -0500 Subject: [PATCH 03/12] fix: include soul field in GET /api/agents list response (fixes #1819) (#1863) Previously, the list endpoint always returned soul=null because _agent_config_to_response() was called without include_soul=True. This caused confusion since PUT /api/agents/{name} and GET /api/agents/{name} both returned the soul content, but the list endpoint silently omitted it. Co-authored-by: octo-patch --- backend/app/gateway/routers/agents.py | 8 ++++---- backend/tests/test_custom_agent.py | 9 +++++++++ 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/backend/app/gateway/routers/agents.py b/backend/app/gateway/routers/agents.py index 00b35857f..ec5e2faac 100644 --- a/backend/app/gateway/routers/agents.py +++ b/backend/app/gateway/routers/agents.py @@ -24,7 +24,7 @@ class AgentResponse(BaseModel): description: str = Field(default="", description="Agent description") model: str | None = Field(default=None, description="Optional model override") tool_groups: list[str] | None = Field(default=None, description="Optional tool group whitelist") - soul: str | None = Field(default=None, description="SOUL.md content (included on GET /{name})") + soul: str | None = Field(default=None, description="SOUL.md content") class AgentsListResponse(BaseModel): @@ -92,17 +92,17 @@ def _agent_config_to_response(agent_cfg: AgentConfig, include_soul: bool = False "/agents", response_model=AgentsListResponse, summary="List Custom Agents", - description="List all custom agents available in the agents directory.", + description="List all custom agents available in the agents directory, including their soul content.", ) async def list_agents() -> AgentsListResponse: """List all custom agents. Returns: - List of all custom agents with their metadata (without soul content). + List of all custom agents with their metadata and soul content. """ try: agents = list_custom_agents() - return AgentsListResponse(agents=[_agent_config_to_response(a) for a in agents]) + return AgentsListResponse(agents=[_agent_config_to_response(a, include_soul=True) for a in agents]) except Exception as e: logger.error(f"Failed to list agents: {e}", exc_info=True) raise HTTPException(status_code=500, detail=f"Failed to list agents: {str(e)}") diff --git a/backend/tests/test_custom_agent.py b/backend/tests/test_custom_agent.py index c97cb4789..9b5e7bb28 100644 --- a/backend/tests/test_custom_agent.py +++ b/backend/tests/test_custom_agent.py @@ -439,6 +439,15 @@ class TestAgentsAPI: assert "agent-one" in names assert "agent-two" in names + def test_list_agents_includes_soul(self, agent_client): + agent_client.post("/api/agents", json={"name": "soul-agent", "soul": "My soul content"}) + + response = agent_client.get("/api/agents") + assert response.status_code == 200 + agents = response.json()["agents"] + soul_agent = next(a for a in agents if a["name"] == "soul-agent") + assert soul_agent["soul"] == "My soul content" + def test_get_agent(self, agent_client): agent_client.post("/api/agents", json={"name": "test-agent", "soul": "Hello world"}) From 72d4347adb269f0c9eb9fcb120d7fb5550a6a103 Mon Sep 17 00:00:00 2001 From: SHIYAO ZHANG <834247613@qq.com> Date: Sun, 5 Apr 2026 10:58:38 +0800 Subject: [PATCH 04/12] fix(sandbox): guard against None runtime.context in sandbox tool helpers (#1853) sandbox_from_runtime() and ensure_sandbox_initialized() write sandbox_id into runtime.context after acquiring a sandbox. When lazy_init=True and no context is supplied to the graph run, runtime.context is None (the LangGraph default), causing a TypeError on the assignment. Add `if runtime.context is not None` guards at all three write sites. Reads already had equivalent guards (e.g. `runtime.context.get(...) if runtime.context else None`); this brings writes into line. --- backend/packages/harness/deerflow/sandbox/tools.py | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/backend/packages/harness/deerflow/sandbox/tools.py b/backend/packages/harness/deerflow/sandbox/tools.py index acd661db0..cad88dc93 100644 --- a/backend/packages/harness/deerflow/sandbox/tools.py +++ b/backend/packages/harness/deerflow/sandbox/tools.py @@ -801,7 +801,8 @@ def sandbox_from_runtime(runtime: ToolRuntime[ContextT, ThreadState] | None = No if sandbox is None: raise SandboxNotFoundError(f"Sandbox with ID '{sandbox_id}' not found", sandbox_id=sandbox_id) - runtime.context["sandbox_id"] = sandbox_id # Ensure sandbox_id is in context for downstream use + if runtime.context is not None: + runtime.context["sandbox_id"] = sandbox_id # Ensure sandbox_id is in context for downstream use return sandbox @@ -836,7 +837,8 @@ def ensure_sandbox_initialized(runtime: ToolRuntime[ContextT, ThreadState] | Non if sandbox_id is not None: sandbox = get_sandbox_provider().get(sandbox_id) if sandbox is not None: - runtime.context["sandbox_id"] = sandbox_id # Ensure sandbox_id is in context for releasing in after_agent + if runtime.context is not None: + runtime.context["sandbox_id"] = sandbox_id # Ensure sandbox_id is in context for releasing in after_agent return sandbox # Sandbox was released, fall through to acquire new one @@ -858,7 +860,8 @@ def ensure_sandbox_initialized(runtime: ToolRuntime[ContextT, ThreadState] | Non if sandbox is None: raise SandboxNotFoundError("Sandbox not found after acquisition", sandbox_id=sandbox_id) - runtime.context["sandbox_id"] = sandbox_id # Ensure sandbox_id is in context for releasing in after_agent + if runtime.context is not None: + runtime.context["sandbox_id"] = sandbox_id # Ensure sandbox_id is in context for releasing in after_agent return sandbox From e5416b539ae9bb2921e4bf58ed2375a4650e20bf Mon Sep 17 00:00:00 2001 From: yangzheli <43645580+yangzheli@users.noreply.github.com> Date: Sun, 5 Apr 2026 15:30:34 +0800 Subject: [PATCH 05/12] fix(docker): use multi-stage build to remove build-essential from runtime image (#1846) * fix(docker): use multi-stage build to remove build-essential from runtime image The build-essential toolchain (~200 MB) was only needed for compiling native Python extensions during `uv sync` but remained in the final image, increasing size and attack surface. Split the Dockerfile into a builder stage (with build-essential) and a clean runtime stage that copies only the compiled artifacts, Node.js, Docker CLI, and uv. Co-Authored-By: Claude Opus 4.6 * fix(docker): add dev stage and pin docker:cli per review feedback Address Copilot review comments: - Add a `dev` build stage (FROM builder) that retains build-essential so startup-time `uv sync` in dev containers can compile from source - Update docker-compose-dev.yaml to use `target: dev` for gateway and langgraph services - Keep the clean runtime stage (no build-essential) as the default final stage for production builds Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 --- backend/Dockerfile | 53 ++++++++++++++++++++++++++++------ docker/docker-compose-dev.yaml | 2 ++ 2 files changed, 46 insertions(+), 9 deletions(-) diff --git a/backend/Dockerfile b/backend/Dockerfile index f4063d7ae..b3e1aea36 100644 --- a/backend/Dockerfile +++ b/backend/Dockerfile @@ -1,10 +1,14 @@ -# Backend Development Dockerfile +# Backend Dockerfile — multi-stage build +# Stage 1 (builder): compiles native Python extensions with build-essential +# Stage 2 (dev): retains toolchain for dev containers (uv sync at startup) +# Stage 3 (runtime): clean image without compiler toolchain for production # UV source image (override for restricted networks that cannot reach ghcr.io) ARG UV_IMAGE=ghcr.io/astral-sh/uv:0.7.20 FROM ${UV_IMAGE} AS uv-source -FROM python:3.12-slim-bookworm +# ── Stage 1: Builder ────────────────────────────────────────────────────────── +FROM python:3.12-slim-bookworm AS builder ARG NODE_MAJOR=22 ARG APT_MIRROR @@ -16,7 +20,7 @@ RUN if [ -n "${APT_MIRROR}" ]; then \ sed -i "s|deb.debian.org|${APT_MIRROR}|g" /etc/apt/sources.list 2>/dev/null || true; \ fi -# Install system dependencies + Node.js (provides npx for MCP servers) +# Install build tools + Node.js (build-essential needed for native Python extensions) RUN apt-get update && apt-get install -y \ curl \ build-essential \ @@ -29,6 +33,41 @@ RUN apt-get update && apt-get install -y \ && apt-get install -y nodejs \ && rm -rf /var/lib/apt/lists/* +# Install uv (source image overridable via UV_IMAGE build arg) +COPY --from=uv-source /uv /uvx /usr/local/bin/ + +# Set working directory +WORKDIR /app + +# Copy backend source code +COPY backend ./backend + +# Install dependencies with cache mount +RUN --mount=type=cache,target=/root/.cache/uv \ + sh -c "cd backend && UV_INDEX_URL=${UV_INDEX_URL:-https://pypi.org/simple} uv sync" + +# ── Stage 2: Dev ────────────────────────────────────────────────────────────── +# Retains compiler toolchain from builder so startup-time `uv sync` can build +# source distributions in development containers. +FROM builder AS dev + +# Install Docker CLI (for DooD: allows starting sandbox containers via host Docker socket) +COPY --from=docker:cli /usr/local/bin/docker /usr/local/bin/docker + +EXPOSE 8001 2024 + +CMD ["sh", "-c", "cd backend && PYTHONPATH=. uv run uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001"] + +# ── Stage 3: Runtime ────────────────────────────────────────────────────────── +# Clean image without build-essential — reduces size (~200 MB) and attack surface. +FROM python:3.12-slim-bookworm + +# Copy Node.js runtime from builder (provides npx for MCP servers) +COPY --from=builder /usr/bin/node /usr/bin/node +COPY --from=builder /usr/lib/node_modules /usr/lib/node_modules +RUN ln -s ../lib/node_modules/npm/bin/npm-cli.js /usr/bin/npm \ + && ln -s ../lib/node_modules/npm/bin/npx-cli.js /usr/bin/npx + # Install Docker CLI (for DooD: allows starting sandbox containers via host Docker socket) COPY --from=docker:cli /usr/local/bin/docker /usr/local/bin/docker @@ -38,12 +77,8 @@ COPY --from=uv-source /uv /uvx /usr/local/bin/ # Set working directory WORKDIR /app -# Copy frontend source code -COPY backend ./backend - -# Install dependencies with cache mount -RUN --mount=type=cache,target=/root/.cache/uv \ - sh -c "cd backend && UV_INDEX_URL=${UV_INDEX_URL:-https://pypi.org/simple} uv sync" +# Copy backend with pre-built virtualenv from builder +COPY --from=builder /app/backend ./backend # Expose ports (gateway: 8001, langgraph: 2024) EXPOSE 8001 2024 diff --git a/docker/docker-compose-dev.yaml b/docker/docker-compose-dev.yaml index 8f5c89b4d..c0749ba9d 100644 --- a/docker/docker-compose-dev.yaml +++ b/docker/docker-compose-dev.yaml @@ -113,6 +113,7 @@ services: build: context: ../ dockerfile: backend/Dockerfile + target: dev # cache_from disabled - requires manual setup: mkdir -p /tmp/docker-cache-gateway args: APT_MIRROR: ${APT_MIRROR:-} @@ -169,6 +170,7 @@ services: build: context: ../ dockerfile: backend/Dockerfile + target: dev # cache_from disabled - requires manual setup: mkdir -p /tmp/docker-cache-langgraph args: APT_MIRROR: ${APT_MIRROR:-} From d3b59a7931e7fd8bd2ef88b590935de50bd77276 Mon Sep 17 00:00:00 2001 From: Echo-Nie <157974576+Echo-Nie@users.noreply.github.com> Date: Sun, 5 Apr 2026 15:35:42 +0800 Subject: [PATCH 06/12] docs: fix some broken links (#1864) * Rename BACKEND_TODO.md to TODO.md in documentation * Update MCP Setup Guide link in CONTRIBUTING.md * Update reference to config.yaml path in documentation * Fix config file path in TITLE_GENERATION_IMPLEMENTATION.md Updated the path to the example config file in the documentation. --- CONTRIBUTING.md | 2 +- backend/docs/AUTO_TITLE_GENERATION.md | 2 +- backend/docs/TITLE_GENERATION_IMPLEMENTATION.md | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 03486cd39..c64a60d2c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -310,7 +310,7 @@ Every pull request runs the backend regression workflow at [.github/workflows/ba - [Configuration Guide](backend/docs/CONFIGURATION.md) - Setup and configuration - [Architecture Overview](backend/CLAUDE.md) - Technical architecture -- [MCP Setup Guide](MCP_SETUP.md) - Model Context Protocol configuration +- [MCP Setup Guide](backend/docs/MCP_SERVER.md) - Model Context Protocol configuration ## Need Help? diff --git a/backend/docs/AUTO_TITLE_GENERATION.md b/backend/docs/AUTO_TITLE_GENERATION.md index 2fd220ea5..27644b2ea 100644 --- a/backend/docs/AUTO_TITLE_GENERATION.md +++ b/backend/docs/AUTO_TITLE_GENERATION.md @@ -248,7 +248,7 @@ def after_agent(self, state: TitleMiddlewareState, runtime: Runtime) -> dict | N - [`packages/harness/deerflow/agents/thread_state.py`](../packages/harness/deerflow/agents/thread_state.py) - ThreadState 定义 - [`packages/harness/deerflow/agents/middlewares/title_middleware.py`](../packages/harness/deerflow/agents/middlewares/title_middleware.py) - TitleMiddleware 实现 - [`packages/harness/deerflow/config/title_config.py`](../packages/harness/deerflow/config/title_config.py) - 配置管理 -- [`config.yaml`](../config.yaml) - 配置文件 +- [`config.yaml`](../../config.example.yaml) - 配置文件 - [`packages/harness/deerflow/agents/lead_agent/agent.py`](../packages/harness/deerflow/agents/lead_agent/agent.py) - Middleware 注册 ## 参考资料 diff --git a/backend/docs/TITLE_GENERATION_IMPLEMENTATION.md b/backend/docs/TITLE_GENERATION_IMPLEMENTATION.md index e4ed673f5..07a026e79 100644 --- a/backend/docs/TITLE_GENERATION_IMPLEMENTATION.md +++ b/backend/docs/TITLE_GENERATION_IMPLEMENTATION.md @@ -30,7 +30,7 @@ ### 2. 配置文件 -#### [`config.yaml`](../config.yaml) +#### [`config.yaml`](../../config.example.yaml) - ✅ 添加 title 配置段： ```yaml title: @@ -51,7 +51,7 @@ title: - ✅ 故障排查指南 - ✅ State vs Metadata 对比 -#### [`BACKEND_TODO.md`](../BACKEND_TODO.md) +#### [`TODO.md`](TODO.md) - ✅ 添加功能完成记录 ### 4. 测试 From 0ffe5a73c1440f2d61d04bc3a16529942d62300e Mon Sep 17 00:00:00 2001 From: Markus Corazzione <83182424+corazzione@users.noreply.github.com> Date: Sun, 5 Apr 2026 04:41:00 -0300 Subject: [PATCH 07/12] chroe(config):Increase subagent max-turn limits (#1852) --- .../deerflow/config/subagents_config.py | 43 +++++- .../deerflow/subagents/builtins/bash_agent.py | 2 +- .../subagents/builtins/general_purpose.py | 2 +- .../harness/deerflow/subagents/registry.py | 22 ++- backend/tests/test_subagent_timeout_config.py | 128 +++++++++++++----- config.example.yaml | 6 +- 6 files changed, 161 insertions(+), 42 deletions(-) diff --git a/backend/packages/harness/deerflow/config/subagents_config.py b/backend/packages/harness/deerflow/config/subagents_config.py index 2611fe8fb..f2c650709 100644 --- a/backend/packages/harness/deerflow/config/subagents_config.py +++ b/backend/packages/harness/deerflow/config/subagents_config.py @@ -15,6 +15,11 @@ class SubagentOverrideConfig(BaseModel): ge=1, description="Timeout in seconds for this subagent (None = use global default)", ) + max_turns: int | None = Field( + default=None, + ge=1, + description="Maximum turns for this subagent (None = use global or builtin default)", + ) class SubagentsAppConfig(BaseModel): @@ -25,6 +30,11 @@ class SubagentsAppConfig(BaseModel): ge=1, description="Default timeout in seconds for all subagents (default: 900 = 15 minutes)", ) + max_turns: int | None = Field( + default=None, + ge=1, + description="Optional default max-turn override for all subagents (None = keep builtin defaults)", + ) agents: dict[str, SubagentOverrideConfig] = Field( default_factory=dict, description="Per-agent configuration overrides keyed by agent name", @@ -44,6 +54,15 @@ class SubagentsAppConfig(BaseModel): return override.timeout_seconds return self.timeout_seconds + def get_max_turns_for(self, agent_name: str, builtin_default: int) -> int: + """Get the effective max_turns for a specific agent.""" + override = self.agents.get(agent_name) + if override is not None and override.max_turns is not None: + return override.max_turns + if self.max_turns is not None: + return self.max_turns + return builtin_default + _subagents_config: SubagentsAppConfig = SubagentsAppConfig() @@ -58,8 +77,26 @@ def load_subagents_config_from_dict(config_dict: dict) -> None: global _subagents_config _subagents_config = SubagentsAppConfig(**config_dict) - overrides_summary = {name: f"{override.timeout_seconds}s" for name, override in _subagents_config.agents.items() if override.timeout_seconds is not None} + overrides_summary = {} + for name, override in _subagents_config.agents.items(): + parts = [] + if override.timeout_seconds is not None: + parts.append(f"timeout={override.timeout_seconds}s") + if override.max_turns is not None: + parts.append(f"max_turns={override.max_turns}") + if parts: + overrides_summary[name] = ", ".join(parts) + if overrides_summary: - logger.info(f"Subagents config loaded: default timeout={_subagents_config.timeout_seconds}s, per-agent overrides={overrides_summary}") + logger.info( + "Subagents config loaded: default timeout=%ss, default max_turns=%s, per-agent overrides=%s", + _subagents_config.timeout_seconds, + _subagents_config.max_turns, + overrides_summary, + ) else: - logger.info(f"Subagents config loaded: default timeout={_subagents_config.timeout_seconds}s, no per-agent overrides") + logger.info( + "Subagents config loaded: default timeout=%ss, default max_turns=%s, no per-agent overrides", + _subagents_config.timeout_seconds, + _subagents_config.max_turns, + ) diff --git a/backend/packages/harness/deerflow/subagents/builtins/bash_agent.py b/backend/packages/harness/deerflow/subagents/builtins/bash_agent.py index 409594efa..094ec65e7 100644 --- a/backend/packages/harness/deerflow/subagents/builtins/bash_agent.py +++ b/backend/packages/harness/deerflow/subagents/builtins/bash_agent.py @@ -43,5 +43,5 @@ You have access to the sandbox environment: tools=["bash", "ls", "read_file", "write_file", "str_replace"], # Sandbox tools only disallowed_tools=["task", "ask_clarification", "present_files"], model="inherit", - max_turns=30, + max_turns=60, ) diff --git a/backend/packages/harness/deerflow/subagents/builtins/general_purpose.py b/backend/packages/harness/deerflow/subagents/builtins/general_purpose.py index 45f1b9fa2..d09d1a00b 100644 --- a/backend/packages/harness/deerflow/subagents/builtins/general_purpose.py +++ b/backend/packages/harness/deerflow/subagents/builtins/general_purpose.py @@ -44,5 +44,5 @@ You have access to the same sandbox environment as the parent agent: tools=None, # Inherit all tools from parent disallowed_tools=["task", "ask_clarification", "present_files"], # Prevent nesting and clarification model="inherit", - max_turns=50, + max_turns=100, ) diff --git a/backend/packages/harness/deerflow/subagents/registry.py b/backend/packages/harness/deerflow/subagents/registry.py index 61da0e453..0192ee7da 100644 --- a/backend/packages/harness/deerflow/subagents/registry.py +++ b/backend/packages/harness/deerflow/subagents/registry.py @@ -28,9 +28,27 @@ def get_subagent_config(name: str) -> SubagentConfig | None: app_config = get_subagents_app_config() effective_timeout = app_config.get_timeout_for(name) + effective_max_turns = app_config.get_max_turns_for(name, config.max_turns) + + overrides = {} if effective_timeout != config.timeout_seconds: - logger.debug(f"Subagent '{name}': timeout overridden by config.yaml ({config.timeout_seconds}s -> {effective_timeout}s)") - config = replace(config, timeout_seconds=effective_timeout) + logger.debug( + "Subagent '%s': timeout overridden by config.yaml (%ss -> %ss)", + name, + config.timeout_seconds, + effective_timeout, + ) + overrides["timeout_seconds"] = effective_timeout + if effective_max_turns != config.max_turns: + logger.debug( + "Subagent '%s': max_turns overridden by config.yaml (%s -> %s)", + name, + config.max_turns, + effective_max_turns, + ) + overrides["max_turns"] = effective_max_turns + if overrides: + config = replace(config, **overrides) return config diff --git a/backend/tests/test_subagent_timeout_config.py b/backend/tests/test_subagent_timeout_config.py index 9edd971a0..50722cc97 100644 --- a/backend/tests/test_subagent_timeout_config.py +++ b/backend/tests/test_subagent_timeout_config.py @@ -1,8 +1,8 @@ -"""Tests for subagent timeout configuration. +"""Tests for subagent runtime configuration. Covers: - SubagentsAppConfig / SubagentOverrideConfig model validation and defaults -- get_timeout_for() resolution logic (global vs per-agent) +- get_timeout_for() / get_max_turns_for() resolution logic - load_subagents_config_from_dict() and get_subagents_app_config() singleton - registry.get_subagent_config() applies config overrides - registry.list_subagents() applies overrides for all agents @@ -24,9 +24,20 @@ from deerflow.subagents.config import SubagentConfig # --------------------------------------------------------------------------- -def _reset_subagents_config(timeout_seconds: int = 900, agents: dict | None = None) -> None: +def _reset_subagents_config( + timeout_seconds: int = 900, + *, + max_turns: int | None = None, + agents: dict | None = None, +) -> None: """Reset global subagents config to a known state.""" - load_subagents_config_from_dict({"timeout_seconds": timeout_seconds, "agents": agents or {}}) + load_subagents_config_from_dict( + { + "timeout_seconds": timeout_seconds, + "max_turns": max_turns, + "agents": agents or {}, + } + ) # --------------------------------------------------------------------------- @@ -38,22 +49,29 @@ class TestSubagentOverrideConfig: def test_default_is_none(self): override = SubagentOverrideConfig() assert override.timeout_seconds is None + assert override.max_turns is None def test_explicit_value(self): - override = SubagentOverrideConfig(timeout_seconds=300) + override = SubagentOverrideConfig(timeout_seconds=300, max_turns=42) assert override.timeout_seconds == 300 + assert override.max_turns == 42 def test_rejects_zero(self): with pytest.raises(ValueError): SubagentOverrideConfig(timeout_seconds=0) + with pytest.raises(ValueError): + SubagentOverrideConfig(max_turns=0) def test_rejects_negative(self): with pytest.raises(ValueError): SubagentOverrideConfig(timeout_seconds=-1) + with pytest.raises(ValueError): + SubagentOverrideConfig(max_turns=-1) def test_minimum_valid_value(self): - override = SubagentOverrideConfig(timeout_seconds=1) + override = SubagentOverrideConfig(timeout_seconds=1, max_turns=1) assert override.timeout_seconds == 1 + assert override.max_turns == 1 # --------------------------------------------------------------------------- @@ -66,66 +84,86 @@ class TestSubagentsAppConfigDefaults: config = SubagentsAppConfig() assert config.timeout_seconds == 900 + def test_default_max_turns_override_is_none(self): + config = SubagentsAppConfig() + assert config.max_turns is None + def test_default_agents_empty(self): config = SubagentsAppConfig() assert config.agents == {} - def test_custom_global_timeout(self): - config = SubagentsAppConfig(timeout_seconds=1800) + def test_custom_global_runtime_overrides(self): + config = SubagentsAppConfig(timeout_seconds=1800, max_turns=120) assert config.timeout_seconds == 1800 + assert config.max_turns == 120 def test_rejects_zero_timeout(self): with pytest.raises(ValueError): SubagentsAppConfig(timeout_seconds=0) + with pytest.raises(ValueError): + SubagentsAppConfig(max_turns=0) def test_rejects_negative_timeout(self): with pytest.raises(ValueError): SubagentsAppConfig(timeout_seconds=-60) + with pytest.raises(ValueError): + SubagentsAppConfig(max_turns=-60) # --------------------------------------------------------------------------- -# SubagentsAppConfig.get_timeout_for() +# SubagentsAppConfig resolution helpers # --------------------------------------------------------------------------- -class TestGetTimeoutFor: +class TestRuntimeResolution: def test_returns_global_default_when_no_override(self): config = SubagentsAppConfig(timeout_seconds=600) assert config.get_timeout_for("general-purpose") == 600 assert config.get_timeout_for("bash") == 600 assert config.get_timeout_for("unknown-agent") == 600 + assert config.get_max_turns_for("general-purpose", 100) == 100 + assert config.get_max_turns_for("bash", 60) == 60 def test_returns_per_agent_override_when_set(self): config = SubagentsAppConfig( timeout_seconds=900, - agents={"bash": SubagentOverrideConfig(timeout_seconds=300)}, + max_turns=120, + agents={"bash": SubagentOverrideConfig(timeout_seconds=300, max_turns=80)}, ) assert config.get_timeout_for("bash") == 300 + assert config.get_max_turns_for("bash", 60) == 80 def test_other_agents_still_use_global_default(self): config = SubagentsAppConfig( timeout_seconds=900, - agents={"bash": SubagentOverrideConfig(timeout_seconds=300)}, + max_turns=140, + agents={"bash": SubagentOverrideConfig(timeout_seconds=300, max_turns=80)}, ) assert config.get_timeout_for("general-purpose") == 900 + assert config.get_max_turns_for("general-purpose", 100) == 140 def test_agent_with_none_override_falls_back_to_global(self): config = SubagentsAppConfig( timeout_seconds=900, - agents={"general-purpose": SubagentOverrideConfig(timeout_seconds=None)}, + max_turns=150, + agents={"general-purpose": SubagentOverrideConfig(timeout_seconds=None, max_turns=None)}, ) assert config.get_timeout_for("general-purpose") == 900 + assert config.get_max_turns_for("general-purpose", 100) == 150 def test_multiple_per_agent_overrides(self): config = SubagentsAppConfig( timeout_seconds=900, + max_turns=120, agents={ - "general-purpose": SubagentOverrideConfig(timeout_seconds=1800), - "bash": SubagentOverrideConfig(timeout_seconds=120), + "general-purpose": SubagentOverrideConfig(timeout_seconds=1800, max_turns=200), + "bash": SubagentOverrideConfig(timeout_seconds=120, max_turns=80), }, ) assert config.get_timeout_for("general-purpose") == 1800 assert config.get_timeout_for("bash") == 120 + assert config.get_max_turns_for("general-purpose", 100) == 200 + assert config.get_max_turns_for("bash", 60) == 80 # --------------------------------------------------------------------------- @@ -139,54 +177,63 @@ class TestLoadSubagentsConfig: _reset_subagents_config() def test_load_global_timeout(self): - load_subagents_config_from_dict({"timeout_seconds": 300}) + load_subagents_config_from_dict({"timeout_seconds": 300, "max_turns": 120}) assert get_subagents_app_config().timeout_seconds == 300 + assert get_subagents_app_config().max_turns == 120 def test_load_with_per_agent_overrides(self): load_subagents_config_from_dict( { "timeout_seconds": 900, + "max_turns": 120, "agents": { - "general-purpose": {"timeout_seconds": 1800}, - "bash": {"timeout_seconds": 60}, + "general-purpose": {"timeout_seconds": 1800, "max_turns": 200}, + "bash": {"timeout_seconds": 60, "max_turns": 80}, }, } ) cfg = get_subagents_app_config() assert cfg.get_timeout_for("general-purpose") == 1800 assert cfg.get_timeout_for("bash") == 60 + assert cfg.get_max_turns_for("general-purpose", 100) == 200 + assert cfg.get_max_turns_for("bash", 60) == 80 def test_load_partial_override(self): load_subagents_config_from_dict( { "timeout_seconds": 600, - "agents": {"bash": {"timeout_seconds": 120}}, + "agents": {"bash": {"timeout_seconds": 120, "max_turns": 70}}, } ) cfg = get_subagents_app_config() assert cfg.get_timeout_for("general-purpose") == 600 assert cfg.get_timeout_for("bash") == 120 + assert cfg.get_max_turns_for("general-purpose", 100) == 100 + assert cfg.get_max_turns_for("bash", 60) == 70 def test_load_empty_dict_uses_defaults(self): load_subagents_config_from_dict({}) cfg = get_subagents_app_config() assert cfg.timeout_seconds == 900 + assert cfg.max_turns is None assert cfg.agents == {} def test_load_replaces_previous_config(self): - load_subagents_config_from_dict({"timeout_seconds": 100}) + load_subagents_config_from_dict({"timeout_seconds": 100, "max_turns": 90}) assert get_subagents_app_config().timeout_seconds == 100 + assert get_subagents_app_config().max_turns == 90 - load_subagents_config_from_dict({"timeout_seconds": 200}) + load_subagents_config_from_dict({"timeout_seconds": 200, "max_turns": 110}) assert get_subagents_app_config().timeout_seconds == 200 + assert get_subagents_app_config().max_turns == 110 def test_singleton_returns_same_instance_between_calls(self): - load_subagents_config_from_dict({"timeout_seconds": 777}) + load_subagents_config_from_dict({"timeout_seconds": 777, "max_turns": 123}) assert get_subagents_app_config() is get_subagents_app_config() # --------------------------------------------------------------------------- -# registry.get_subagent_config – timeout override applied +# registry.get_subagent_config – runtime overrides applied # --------------------------------------------------------------------------- @@ -211,25 +258,29 @@ class TestRegistryGetSubagentConfig: _reset_subagents_config(timeout_seconds=900) config = get_subagent_config("general-purpose") assert config.timeout_seconds == 900 + assert config.max_turns == 100 def test_global_timeout_override_applied(self): from deerflow.subagents.registry import get_subagent_config - _reset_subagents_config(timeout_seconds=1800) + _reset_subagents_config(timeout_seconds=1800, max_turns=140) config = get_subagent_config("general-purpose") assert config.timeout_seconds == 1800 + assert config.max_turns == 140 - def test_per_agent_timeout_override_applied(self): + def test_per_agent_runtime_override_applied(self): from deerflow.subagents.registry import get_subagent_config load_subagents_config_from_dict( { "timeout_seconds": 900, - "agents": {"bash": {"timeout_seconds": 120}}, + "max_turns": 120, + "agents": {"bash": {"timeout_seconds": 120, "max_turns": 80}}, } ) bash_config = get_subagent_config("bash") assert bash_config.timeout_seconds == 120 + assert bash_config.max_turns == 80 def test_per_agent_override_does_not_affect_other_agents(self): from deerflow.subagents.registry import get_subagent_config @@ -237,11 +288,13 @@ class TestRegistryGetSubagentConfig: load_subagents_config_from_dict( { "timeout_seconds": 900, - "agents": {"bash": {"timeout_seconds": 120}}, + "max_turns": 120, + "agents": {"bash": {"timeout_seconds": 120, "max_turns": 80}}, } ) gp_config = get_subagent_config("general-purpose") assert gp_config.timeout_seconds == 900 + assert gp_config.max_turns == 120 def test_builtin_config_object_is_not_mutated(self): """Registry must return a new object, leaving the builtin default intact.""" @@ -249,24 +302,27 @@ class TestRegistryGetSubagentConfig: from deerflow.subagents.registry import get_subagent_config original_timeout = BUILTIN_SUBAGENTS["bash"].timeout_seconds - load_subagents_config_from_dict({"timeout_seconds": 42}) + original_max_turns = BUILTIN_SUBAGENTS["bash"].max_turns + load_subagents_config_from_dict({"timeout_seconds": 42, "max_turns": 88}) returned = get_subagent_config("bash") assert returned.timeout_seconds == 42 + assert returned.max_turns == 88 assert BUILTIN_SUBAGENTS["bash"].timeout_seconds == original_timeout + assert BUILTIN_SUBAGENTS["bash"].max_turns == original_max_turns def test_config_preserves_other_fields(self): - """Applying timeout override must not change other SubagentConfig fields.""" + """Applying runtime overrides must not change other SubagentConfig fields.""" from deerflow.subagents.builtins import BUILTIN_SUBAGENTS from deerflow.subagents.registry import get_subagent_config - _reset_subagents_config(timeout_seconds=300) + _reset_subagents_config(timeout_seconds=300, max_turns=140) original = BUILTIN_SUBAGENTS["general-purpose"] overridden = get_subagent_config("general-purpose") assert overridden.name == original.name assert overridden.description == original.description - assert overridden.max_turns == original.max_turns + assert overridden.max_turns == 140 assert overridden.model == original.model assert overridden.tools == original.tools assert overridden.disallowed_tools == original.disallowed_tools @@ -291,9 +347,10 @@ class TestRegistryListSubagents: def test_all_returned_configs_get_global_override(self): from deerflow.subagents.registry import list_subagents - _reset_subagents_config(timeout_seconds=123) + _reset_subagents_config(timeout_seconds=123, max_turns=77) for cfg in list_subagents(): assert cfg.timeout_seconds == 123, f"{cfg.name} has wrong timeout" + assert cfg.max_turns == 77, f"{cfg.name} has wrong max_turns" def test_per_agent_overrides_reflected_in_list(self): from deerflow.subagents.registry import list_subagents @@ -301,15 +358,18 @@ class TestRegistryListSubagents: load_subagents_config_from_dict( { "timeout_seconds": 900, + "max_turns": 120, "agents": { - "general-purpose": {"timeout_seconds": 1800}, - "bash": {"timeout_seconds": 60}, + "general-purpose": {"timeout_seconds": 1800, "max_turns": 200}, + "bash": {"timeout_seconds": 60, "max_turns": 80}, }, } ) by_name = {cfg.name: cfg for cfg in list_subagents()} assert by_name["general-purpose"].timeout_seconds == 1800 assert by_name["bash"].timeout_seconds == 60 + assert by_name["general-purpose"].max_turns == 200 + assert by_name["bash"].max_turns == 80 # --------------------------------------------------------------------------- diff --git a/config.example.yaml b/config.example.yaml index f68a574e5..d6f382591 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -456,13 +456,17 @@ sandbox: # subagents: # # Default timeout in seconds for all subagents (default: 900 = 15 minutes) # timeout_seconds: 900 +# # Optional global max-turn override for all subagents +# # max_turns: 120 # -# # Optional per-agent timeout overrides +# # Optional per-agent overrides # agents: # general-purpose: # timeout_seconds: 1800 # 30 minutes for complex multi-step tasks +# max_turns: 160 # bash: # timeout_seconds: 300 # 5 minutes for quick command execution +# max_turns: 80 # ============================================================================ # ACP Agents Configuration From 9ca68ffaaa00e53784c84cc9c6cd18144e33efc4 Mon Sep 17 00:00:00 2001 From: Evan Wu <850123119@qq.com> Date: Sun, 5 Apr 2026 15:52:22 +0800 Subject: [PATCH 08/12] fix: preserve virtual path separator style (#1828) * fix: preserve virtual path separator style * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Willem Jiang Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- .../harness/deerflow/sandbox/tools.py | 16 +++-- backend/tests/test_sandbox_tools_security.py | 63 +++++++++++++++++++ 2 files changed, 75 insertions(+), 4 deletions(-) diff --git a/backend/packages/harness/deerflow/sandbox/tools.py b/backend/packages/harness/deerflow/sandbox/tools.py index cad88dc93..b52131ff4 100644 --- a/backend/packages/harness/deerflow/sandbox/tools.py +++ b/backend/packages/harness/deerflow/sandbox/tools.py @@ -366,12 +366,17 @@ def _path_variants(path: str) -> set[str]: return {path, path.replace("\\", "/"), path.replace("/", "\\")} +def _path_separator_for_style(path: str) -> str: + return "\\" if "\\" in path and "/" not in path else "/" + + def _join_path_preserving_style(base: str, relative: str) -> str: if not relative: return base - if "/" in base and "\\" not in base: - return f"{base.rstrip('/')}/{relative}" - return str(Path(base) / relative) + separator = _path_separator_for_style(base) + normalized_relative = relative.replace("\\" if separator == "/" else "/", separator).lstrip("/\\") + stripped_base = base.rstrip("/\\") + return f"{stripped_base}{separator}{normalized_relative}" def _sanitize_error(error: Exception, runtime: "ToolRuntime[ContextT, ThreadState] | None" = None) -> str: @@ -416,7 +421,10 @@ def replace_virtual_path(path: str, thread_data: ThreadDataState | None) -> str: return actual_base if path.startswith(f"{virtual_base}/"): rest = path[len(virtual_base) :].lstrip("/") - return _join_path_preserving_style(actual_base, rest) + result = _join_path_preserving_style(actual_base, rest) + if path.endswith("/") and not result.endswith(("/", "\\")): + result += _path_separator_for_style(actual_base) + return result return path diff --git a/backend/tests/test_sandbox_tools_security.py b/backend/tests/test_sandbox_tools_security.py index 01aedf6be..268c5aada 100644 --- a/backend/tests/test_sandbox_tools_security.py +++ b/backend/tests/test_sandbox_tools_security.py @@ -42,6 +42,53 @@ def test_replace_virtual_path_maps_virtual_root_and_subpaths() -> None: assert Path(replace_virtual_path("/mnt/user-data", _THREAD_DATA)).as_posix() == "/tmp/deer-flow/threads/t1/user-data" +def test_replace_virtual_path_preserves_trailing_slash() -> None: + """Trailing slash must survive virtual-to-actual path replacement. + + Regression: '/mnt/user-data/workspace/' was previously returned without + the trailing slash, causing string concatenations like + output_dir + 'file.txt' to produce a missing-separator path. + """ + result = replace_virtual_path("/mnt/user-data/workspace/", _THREAD_DATA) + assert result.endswith("/"), f"Expected trailing slash, got: {result!r}" + assert result == "/tmp/deer-flow/threads/t1/user-data/workspace/" + + +def test_replace_virtual_path_preserves_trailing_slash_windows_style() -> None: + """Trailing slash must be preserved as backslash when actual_base is Windows-style. + + If actual_base uses backslash separators, appending '/' would produce a + mixed-separator path. The separator must match the style of actual_base. + """ + win_thread_data = { + "workspace_path": r"C:\deer-flow\threads\t1\user-data\workspace", + "uploads_path": r"C:\deer-flow\threads\t1\user-data\uploads", + "outputs_path": r"C:\deer-flow\threads\t1\user-data\outputs", + } + result = replace_virtual_path("/mnt/user-data/workspace/", win_thread_data) + assert result.endswith("\\"), f"Expected trailing backslash for Windows path, got: {result!r}" + assert "/" not in result, f"Mixed separators in Windows path: {result!r}" + + +def test_replace_virtual_path_preserves_windows_style_for_nested_subdir_trailing_slash() -> None: + """Nested Windows-style subdirectories must keep backslashes throughout.""" + win_thread_data = { + "workspace_path": r"C:\deer-flow\threads\t1\user-data\workspace", + "uploads_path": r"C:\deer-flow\threads\t1\user-data\uploads", + "outputs_path": r"C:\deer-flow\threads\t1\user-data\outputs", + } + result = replace_virtual_path("/mnt/user-data/workspace/subdir/", win_thread_data) + assert result == "C:\\deer-flow\\threads\\t1\\user-data\\workspace\\subdir\\" + assert "/" not in result, f"Mixed separators in Windows path: {result!r}" + + +def test_replace_virtual_paths_in_command_preserves_trailing_slash() -> None: + """Trailing slash on a virtual path inside a command must be preserved.""" + cmd = """python -c "output_dir = '/mnt/user-data/workspace/'; print(output_dir + 'some_file.txt')\"""" + result = replace_virtual_paths_in_command(cmd, _THREAD_DATA) + assert "/tmp/deer-flow/threads/t1/user-data/workspace/" in result, f"Trailing slash lost in: {result!r}" + + # ---------- mask_local_paths_in_output ---------- @@ -257,6 +304,22 @@ def test_validate_local_bash_command_paths_blocks_host_paths() -> None: validate_local_bash_command_paths("cat /etc/passwd", _THREAD_DATA) +def test_validate_local_bash_command_paths_allows_https_urls() -> None: + """URLs like https://github.com/... must not be flagged as unsafe absolute paths.""" + validate_local_bash_command_paths( + "cd /mnt/user-data/workspace && git clone https://github.com/CherryHQ/cherry-studio.git", + _THREAD_DATA, + ) + + +def test_validate_local_bash_command_paths_allows_http_urls() -> None: + """HTTP URLs must not be flagged as unsafe absolute paths.""" + validate_local_bash_command_paths( + "curl http://example.com/file.tar.gz -o /mnt/user-data/workspace/file.tar.gz", + _THREAD_DATA, + ) + + def test_validate_local_bash_command_paths_allows_virtual_and_system_paths() -> None: validate_local_bash_command_paths( "/bin/echo ok > /mnt/user-data/workspace/out.txt && cat /dev/null", From 8049785de666ddeb143693df21e15d76582acba8 Mon Sep 17 00:00:00 2001 From: thefoolgy <99605054+thefoolgy@users.noreply.github.com> Date: Sun, 5 Apr 2026 16:23:00 +0800 Subject: [PATCH 09/12] fix(memory): case-insensitive fact deduplication and positive reinforcement detection (#1804) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(memory): case-insensitive fact deduplication and positive reinforcement detection Two fixes to the memory system: 1. _fact_content_key() now lowercases content before comparison, preventing semantically duplicate facts like "User prefers Python" and "user prefers python" from being stored separately. 2. Adds detect_reinforcement() to MemoryMiddleware (closes #1719), mirroring detect_correction(). When users signal approval ("yes exactly", "perfect", "完全正确", etc.), the memory updater now receives reinforcement_detected=True and injects a hint prompting the LLM to record confirmed preferences and behaviors with high confidence. Changes across the full signal path: - memory_middleware.py: _REINFORCEMENT_PATTERNS + detect_reinforcement() - queue.py: reinforcement_detected field in ConversationContext and add() - updater.py: reinforcement_detected param in update_memory() and update_memory_from_conversation(); builds reinforcement_hint alongside the existing correction_hint Tests: 11 new tests covering deduplication, hint injection, and signal detection (Chinese + English patterns, window boundary, conflict with correction). Co-Authored-By: Claude Sonnet 4.6 * fix(memory): address Copilot review comments on reinforcement detection - Tighten _REINFORCEMENT_PATTERNS: remove 很好, require punctuation/end-of-string boundaries on remaining patterns, split this-is-good into stricter variants - Suppress reinforcement_detected when correction_detected is true to avoid mixed-signal noise - Use casefold() instead of lower() for Unicode-aware fact deduplication - Add missing test coverage for reinforcement_detected OR merge and forwarding in queue --------- Co-authored-by: Claude Sonnet 4.6 --- .../harness/deerflow/agents/memory/queue.py | 6 + .../harness/deerflow/agents/memory/updater.py | 16 +- .../agents/middlewares/memory_middleware.py | 41 +++++ backend/tests/test_memory_queue.py | 41 +++++ backend/tests/test_memory_updater.py | 153 ++++++++++++++++++ backend/tests/test_memory_upload_filtering.py | 72 ++++++++- 6 files changed, 326 insertions(+), 3 deletions(-) diff --git a/backend/packages/harness/deerflow/agents/memory/queue.py b/backend/packages/harness/deerflow/agents/memory/queue.py index 6d777a67e..d78c643f8 100644 --- a/backend/packages/harness/deerflow/agents/memory/queue.py +++ b/backend/packages/harness/deerflow/agents/memory/queue.py @@ -21,6 +21,7 @@ class ConversationContext: timestamp: datetime = field(default_factory=datetime.utcnow) agent_name: str | None = None correction_detected: bool = False + reinforcement_detected: bool = False class MemoryUpdateQueue: @@ -44,6 +45,7 @@ class MemoryUpdateQueue: messages: list[Any], agent_name: str | None = None, correction_detected: bool = False, + reinforcement_detected: bool = False, ) -> None: """Add a conversation to the update queue. @@ -52,6 +54,7 @@ class MemoryUpdateQueue: messages: The conversation messages. agent_name: If provided, memory is stored per-agent. If None, uses global memory. correction_detected: Whether recent turns include an explicit correction signal. + reinforcement_detected: Whether recent turns include a positive reinforcement signal. """ config = get_memory_config() if not config.enabled: @@ -63,11 +66,13 @@ class MemoryUpdateQueue: None, ) merged_correction_detected = correction_detected or (existing_context.correction_detected if existing_context is not None else False) + merged_reinforcement_detected = reinforcement_detected or (existing_context.reinforcement_detected if existing_context is not None else False) context = ConversationContext( thread_id=thread_id, messages=messages, agent_name=agent_name, correction_detected=merged_correction_detected, + reinforcement_detected=merged_reinforcement_detected, ) # Check if this thread already has a pending update @@ -130,6 +135,7 @@ class MemoryUpdateQueue: thread_id=context.thread_id, agent_name=context.agent_name, correction_detected=context.correction_detected, + reinforcement_detected=context.reinforcement_detected, ) if success: logger.info("Memory updated successfully for thread %s", context.thread_id) diff --git a/backend/packages/harness/deerflow/agents/memory/updater.py b/backend/packages/harness/deerflow/agents/memory/updater.py index c59749d7b..5f459b47a 100644 --- a/backend/packages/harness/deerflow/agents/memory/updater.py +++ b/backend/packages/harness/deerflow/agents/memory/updater.py @@ -246,7 +246,7 @@ def _fact_content_key(content: Any) -> str | None: stripped = content.strip() if not stripped: return None - return stripped + return stripped.casefold() class MemoryUpdater: @@ -272,6 +272,7 @@ class MemoryUpdater: thread_id: str | None = None, agent_name: str | None = None, correction_detected: bool = False, + reinforcement_detected: bool = False, ) -> bool: """Update memory based on conversation messages. @@ -280,6 +281,7 @@ class MemoryUpdater: thread_id: Optional thread ID for tracking source. agent_name: If provided, updates per-agent memory. If None, updates global memory. correction_detected: Whether recent turns include an explicit correction signal. + reinforcement_detected: Whether recent turns include a positive reinforcement signal. Returns: True if update was successful, False otherwise. @@ -310,6 +312,14 @@ class MemoryUpdater: "and record the correct approach as a fact with category " '"correction" and confidence >= 0.95 when appropriate.' ) + if reinforcement_detected: + reinforcement_hint = ( + "IMPORTANT: Positive reinforcement signals were detected in this conversation. " + "The user explicitly confirmed the agent's approach was correct or helpful. " + "Record the confirmed approach, style, or preference as a fact with category " + '"preference" or "behavior" and confidence >= 0.9 when appropriate.' + ) + correction_hint = (correction_hint + "\n" + reinforcement_hint).strip() if correction_hint else reinforcement_hint prompt = MEMORY_UPDATE_PROMPT.format( current_memory=json.dumps(current_memory, indent=2), @@ -441,6 +451,7 @@ def update_memory_from_conversation( thread_id: str | None = None, agent_name: str | None = None, correction_detected: bool = False, + reinforcement_detected: bool = False, ) -> bool: """Convenience function to update memory from a conversation. @@ -449,9 +460,10 @@ def update_memory_from_conversation( thread_id: Optional thread ID. agent_name: If provided, updates per-agent memory. If None, updates global memory. correction_detected: Whether recent turns include an explicit correction signal. + reinforcement_detected: Whether recent turns include a positive reinforcement signal. Returns: True if successful, False otherwise. """ updater = MemoryUpdater() - return updater.update_memory(messages, thread_id, agent_name, correction_detected) + return updater.update_memory(messages, thread_id, agent_name, correction_detected, reinforcement_detected) diff --git a/backend/packages/harness/deerflow/agents/middlewares/memory_middleware.py b/backend/packages/harness/deerflow/agents/middlewares/memory_middleware.py index 6215a2957..5e8ca6344 100644 --- a/backend/packages/harness/deerflow/agents/middlewares/memory_middleware.py +++ b/backend/packages/harness/deerflow/agents/middlewares/memory_middleware.py @@ -29,6 +29,22 @@ _CORRECTION_PATTERNS = ( re.compile(r"改用"), ) +_REINFORCEMENT_PATTERNS = ( + re.compile(r"\byes[,.]?\s+(?:exactly|perfect|that(?:'s| is) (?:right|correct|it))\b", re.IGNORECASE), + re.compile(r"\bperfect(?:[.!?]|$)", re.IGNORECASE), + re.compile(r"\bexactly\s+(?:right|correct)\b", re.IGNORECASE), + re.compile(r"\bthat(?:'s| is)\s+(?:exactly\s+)?(?:right|correct|what i (?:wanted|needed|meant))\b", re.IGNORECASE), + re.compile(r"\bkeep\s+(?:doing\s+)?that\b", re.IGNORECASE), + re.compile(r"\bjust\s+(?:like\s+)?(?:that|this)\b", re.IGNORECASE), + re.compile(r"\bthis is (?:great|helpful)\b(?:[.!?]|$)", re.IGNORECASE), + re.compile(r"\bthis is what i wanted\b(?:[.!?]|$)", re.IGNORECASE), + re.compile(r"对[，,]?\s*就是这样(?:[。！？!?.]|$)"), + re.compile(r"完全正确(?:[。！？!?.]|$)"), + re.compile(r"(?:对[，,]?\s*)?就是这个意思(?:[。！？!?.]|$)"), + re.compile(r"正是我想要的(?:[。！？!?.]|$)"), + re.compile(r"继续保持(?:[。！？!?.]|$)"), +) + class MemoryMiddlewareState(AgentState): """Compatible with the `ThreadState` schema.""" @@ -132,6 +148,29 @@ def detect_correction(messages: list[Any]) -> bool: return False +def detect_reinforcement(messages: list[Any]) -> bool: + """Detect explicit positive reinforcement signals in recent conversation turns. + + Complements detect_correction() by identifying when the user confirms the + agent's approach was correct. This allows the memory system to record what + worked well, not just what went wrong. + + The queue keeps only one pending context per thread, so callers pass the + latest filtered message list. Checking only recent user turns keeps signal + detection conservative while avoiding stale signals from long histories. + """ + recent_user_msgs = [msg for msg in messages[-6:] if getattr(msg, "type", None) == "human"] + + for msg in recent_user_msgs: + content = _extract_message_text(msg).strip() + if not content: + continue + if any(pattern.search(content) for pattern in _REINFORCEMENT_PATTERNS): + return True + + return False + + class MemoryMiddleware(AgentMiddleware[MemoryMiddlewareState]): """Middleware that queues conversation for memory update after agent execution. @@ -196,12 +235,14 @@ class MemoryMiddleware(AgentMiddleware[MemoryMiddlewareState]): # Queue the filtered conversation for memory update correction_detected = detect_correction(filtered_messages) + reinforcement_detected = not correction_detected and detect_reinforcement(filtered_messages) queue = get_memory_queue() queue.add( thread_id=thread_id, messages=filtered_messages, agent_name=self._agent_name, correction_detected=correction_detected, + reinforcement_detected=reinforcement_detected, ) return None diff --git a/backend/tests/test_memory_queue.py b/backend/tests/test_memory_queue.py index 6ef91a142..204f9d16e 100644 --- a/backend/tests/test_memory_queue.py +++ b/backend/tests/test_memory_queue.py @@ -47,4 +47,45 @@ def test_process_queue_forwards_correction_flag_to_updater() -> None: thread_id="thread-1", agent_name="lead_agent", correction_detected=True, + reinforcement_detected=False, + ) + + +def test_queue_add_preserves_existing_reinforcement_flag_for_same_thread() -> None: + queue = MemoryUpdateQueue() + + with ( + patch("deerflow.agents.memory.queue.get_memory_config", return_value=_memory_config(enabled=True)), + patch.object(queue, "_reset_timer"), + ): + queue.add(thread_id="thread-1", messages=["first"], reinforcement_detected=True) + queue.add(thread_id="thread-1", messages=["second"], reinforcement_detected=False) + + assert len(queue._queue) == 1 + assert queue._queue[0].messages == ["second"] + assert queue._queue[0].reinforcement_detected is True + + +def test_process_queue_forwards_reinforcement_flag_to_updater() -> None: + queue = MemoryUpdateQueue() + queue._queue = [ + ConversationContext( + thread_id="thread-1", + messages=["conversation"], + agent_name="lead_agent", + reinforcement_detected=True, + ) + ] + mock_updater = MagicMock() + mock_updater.update_memory.return_value = True + + with patch("deerflow.agents.memory.updater.MemoryUpdater", return_value=mock_updater): + queue._process_queue() + + mock_updater.update_memory.assert_called_once_with( + messages=["conversation"], + thread_id="thread-1", + agent_name="lead_agent", + correction_detected=False, + reinforcement_detected=True, ) diff --git a/backend/tests/test_memory_updater.py b/backend/tests/test_memory_updater.py index 6309cf9f6..48fdfd89e 100644 --- a/backend/tests/test_memory_updater.py +++ b/backend/tests/test_memory_updater.py @@ -619,3 +619,156 @@ class TestUpdateMemoryStructuredResponse: assert result is True prompt = model.invoke.call_args[0][0] assert "Explicit correction signals were detected" not in prompt + + +class TestFactDeduplicationCaseInsensitive: + """Tests that fact deduplication is case-insensitive.""" + + def test_duplicate_fact_different_case_not_stored(self): + updater = MemoryUpdater() + current_memory = _make_memory( + facts=[ + { + "id": "fact_1", + "content": "User prefers Python", + "category": "preference", + "confidence": 0.9, + "createdAt": "2026-01-01T00:00:00Z", + "source": "thread-a", + }, + ] + ) + # Same fact with different casing should be treated as duplicate + update_data = { + "factsToRemove": [], + "newFacts": [ + {"content": "user prefers python", "category": "preference", "confidence": 0.95}, + ], + } + + with patch( + "deerflow.agents.memory.updater.get_memory_config", + return_value=_memory_config(max_facts=100, fact_confidence_threshold=0.7), + ): + result = updater._apply_updates(current_memory, update_data, thread_id="thread-b") + + # Should still have only 1 fact (duplicate rejected) + assert len(result["facts"]) == 1 + assert result["facts"][0]["content"] == "User prefers Python" + + def test_unique_fact_different_case_and_content_stored(self): + updater = MemoryUpdater() + current_memory = _make_memory( + facts=[ + { + "id": "fact_1", + "content": "User prefers Python", + "category": "preference", + "confidence": 0.9, + "createdAt": "2026-01-01T00:00:00Z", + "source": "thread-a", + }, + ] + ) + update_data = { + "factsToRemove": [], + "newFacts": [ + {"content": "User prefers Go", "category": "preference", "confidence": 0.85}, + ], + } + + with patch( + "deerflow.agents.memory.updater.get_memory_config", + return_value=_memory_config(max_facts=100, fact_confidence_threshold=0.7), + ): + result = updater._apply_updates(current_memory, update_data, thread_id="thread-b") + + assert len(result["facts"]) == 2 + + +class TestReinforcementHint: + """Tests that reinforcement_detected injects the correct hint into the prompt.""" + + @staticmethod + def _make_mock_model(json_response: str): + model = MagicMock() + response = MagicMock() + response.content = f"```json\n{json_response}\n```" + model.invoke.return_value = response + return model + + def test_reinforcement_hint_injected_when_detected(self): + updater = MemoryUpdater() + valid_json = '{"user": {}, "history": {}, "newFacts": [], "factsToRemove": []}' + model = self._make_mock_model(valid_json) + + with ( + patch.object(updater, "_get_model", return_value=model), + patch("deerflow.agents.memory.updater.get_memory_config", return_value=_memory_config(enabled=True)), + patch("deerflow.agents.memory.updater.get_memory_data", return_value=_make_memory()), + patch("deerflow.agents.memory.updater.get_memory_storage", return_value=MagicMock(save=MagicMock(return_value=True))), + ): + msg = MagicMock() + msg.type = "human" + msg.content = "Yes, exactly! That's what I needed." + ai_msg = MagicMock() + ai_msg.type = "ai" + ai_msg.content = "Great to hear!" + ai_msg.tool_calls = [] + + result = updater.update_memory([msg, ai_msg], reinforcement_detected=True) + + assert result is True + prompt = model.invoke.call_args[0][0] + assert "Positive reinforcement signals were detected" in prompt + + def test_reinforcement_hint_absent_when_not_detected(self): + updater = MemoryUpdater() + valid_json = '{"user": {}, "history": {}, "newFacts": [], "factsToRemove": []}' + model = self._make_mock_model(valid_json) + + with ( + patch.object(updater, "_get_model", return_value=model), + patch("deerflow.agents.memory.updater.get_memory_config", return_value=_memory_config(enabled=True)), + patch("deerflow.agents.memory.updater.get_memory_data", return_value=_make_memory()), + patch("deerflow.agents.memory.updater.get_memory_storage", return_value=MagicMock(save=MagicMock(return_value=True))), + ): + msg = MagicMock() + msg.type = "human" + msg.content = "Tell me more." + ai_msg = MagicMock() + ai_msg.type = "ai" + ai_msg.content = "Sure." + ai_msg.tool_calls = [] + + result = updater.update_memory([msg, ai_msg], reinforcement_detected=False) + + assert result is True + prompt = model.invoke.call_args[0][0] + assert "Positive reinforcement signals were detected" not in prompt + + def test_both_hints_present_when_both_detected(self): + updater = MemoryUpdater() + valid_json = '{"user": {}, "history": {}, "newFacts": [], "factsToRemove": []}' + model = self._make_mock_model(valid_json) + + with ( + patch.object(updater, "_get_model", return_value=model), + patch("deerflow.agents.memory.updater.get_memory_config", return_value=_memory_config(enabled=True)), + patch("deerflow.agents.memory.updater.get_memory_data", return_value=_make_memory()), + patch("deerflow.agents.memory.updater.get_memory_storage", return_value=MagicMock(save=MagicMock(return_value=True))), + ): + msg = MagicMock() + msg.type = "human" + msg.content = "No wait, that's wrong. Actually yes, exactly right." + ai_msg = MagicMock() + ai_msg.type = "ai" + ai_msg.content = "Got it." + ai_msg.tool_calls = [] + + result = updater.update_memory([msg, ai_msg], correction_detected=True, reinforcement_detected=True) + + assert result is True + prompt = model.invoke.call_args[0][0] + assert "Explicit correction signals were detected" in prompt + assert "Positive reinforcement signals were detected" in prompt diff --git a/backend/tests/test_memory_upload_filtering.py b/backend/tests/test_memory_upload_filtering.py index 1ff0aa3b6..2e2308b61 100644 --- a/backend/tests/test_memory_upload_filtering.py +++ b/backend/tests/test_memory_upload_filtering.py @@ -10,7 +10,7 @@ persisting in long-term memory: from langchain_core.messages import AIMessage, HumanMessage, ToolMessage from deerflow.agents.memory.updater import _strip_upload_mentions_from_memory -from deerflow.agents.middlewares.memory_middleware import _filter_messages_for_memory, detect_correction +from deerflow.agents.middlewares.memory_middleware import _filter_messages_for_memory, detect_correction, detect_reinforcement # --------------------------------------------------------------------------- # Helpers @@ -270,3 +270,73 @@ class TestStripUploadMentionsFromMemory: mem = {"user": {}, "history": {}, "facts": []} result = _strip_upload_mentions_from_memory(mem) assert result == {"user": {}, "history": {}, "facts": []} + + +# =========================================================================== +# detect_reinforcement +# =========================================================================== + + +class TestDetectReinforcement: + def test_detects_english_reinforcement_signal(self): + msgs = [ + _human("Can you summarise it in bullet points?"), + _ai("Here are the key points: ..."), + _human("Yes, exactly! That's what I needed."), + _ai("Glad it helped."), + ] + + assert detect_reinforcement(msgs) is True + + def test_detects_perfect_signal(self): + msgs = [ + _human("Write it more concisely."), + _ai("Here is the concise version."), + _human("Perfect."), + _ai("Great!"), + ] + + assert detect_reinforcement(msgs) is True + + def test_detects_chinese_reinforcement_signal(self): + msgs = [ + _human("帮我用要点来总结"), + _ai("好的，要点如下：..."), + _human("完全正确，就是这个意思"), + _ai("很高兴能帮到你"), + ] + + assert detect_reinforcement(msgs) is True + + def test_returns_false_without_signal(self): + msgs = [ + _human("What does this function do?"), + _ai("It processes the input data."), + _human("Can you show me an example?"), + ] + + assert detect_reinforcement(msgs) is False + + def test_only_checks_recent_messages(self): + # Reinforcement signal buried beyond the -6 window should not trigger + msgs = [ + _human("Yes, exactly right."), + _ai("Noted."), + _human("Let's discuss tests."), + _ai("Sure."), + _human("What about linting?"), + _ai("Use ruff."), + _human("And formatting?"), + _ai("Use make format."), + ] + + assert detect_reinforcement(msgs) is False + + def test_does_not_conflict_with_correction(self): + # A message can trigger correction but not reinforcement + msgs = [ + _human("That's wrong, try again."), + _ai("Corrected."), + ] + + assert detect_reinforcement(msgs) is False From 28474c47cbea8db2fcd62a651996eb875bde710c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=82=96?= <168966994+luoxiao6645@users.noreply.github.com> Date: Sun, 5 Apr 2026 16:35:33 +0800 Subject: [PATCH 10/12] fix: avoid command palette hydration mismatch on macOS (#1563) # Conflicts: # frontend/src/components/workspace/command-palette.tsx Co-authored-by: luoxiao6645 Co-authored-by: Willem Jiang --- .../src/components/workspace/command-palette.tsx | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/frontend/src/components/workspace/command-palette.tsx b/frontend/src/components/workspace/command-palette.tsx index 915410b37..1d754f61d 100644 --- a/frontend/src/components/workspace/command-palette.tsx +++ b/frontend/src/components/workspace/command-palette.tsx @@ -32,14 +32,10 @@ import { SettingsDialog } from "./settings"; export function CommandPalette() { const { t } = useI18n(); const router = useRouter(); - const [mounted, setMounted] = useState(false); const [open, setOpen] = useState(false); const [shortcutsOpen, setShortcutsOpen] = useState(false); const [settingsOpen, setSettingsOpen] = useState(false); - - useEffect(() => { - setMounted(true); - }, []); + const [isMac, setIsMac] = useState(false); const handleNewChat = useCallback(() => { router.push("/workspace/chats/new"); @@ -68,14 +64,12 @@ export function CommandPalette() { useGlobalShortcuts(shortcuts); - const isMac = mounted && navigator.userAgent.includes("Mac"); + useEffect(() => { + setIsMac(navigator.userAgent.includes("Mac")); + }, []); const metaKey = isMac ? "⌘" : "Ctrl+"; const shiftKey = isMac ? "⇧" : "Shift+"; - if (!mounted) { - return null; - } - return ( <> From 117fa9b05d7061cb7733c04e24c7fe190ab21510 Mon Sep 17 00:00:00 2001 From: Chris Z <535257617@qq.com> Date: Sun, 5 Apr 2026 18:04:21 +0800 Subject: [PATCH 11/12] fix(channels): normalize slack allowed user ids (#1802) * fix(channels): normalize slack allowed user ids * style(channels): apply backend formatter --------- Co-authored-by: haimingZZ <15558128926@qq.com> Co-authored-by: suyua9 <1521777066@qq.com> --- backend/app/channels/slack.py | 2 +- backend/tests/test_channels.py | 43 +++++++++++++++++++++++++++++++++- 2 files changed, 43 insertions(+), 2 deletions(-) diff --git a/backend/app/channels/slack.py b/backend/app/channels/slack.py index 32b42e5a8..c9ad6a6ec 100644 --- a/backend/app/channels/slack.py +++ b/backend/app/channels/slack.py @@ -30,7 +30,7 @@ class SlackChannel(Channel): self._socket_client = None self._web_client = None self._loop: asyncio.AbstractEventLoop | None = None - self._allowed_users: set[str] = set(config.get("allowed_users", [])) + self._allowed_users: set[str] = {str(user_id) for user_id in config.get("allowed_users", [])} async def start(self) -> None: if self._running: diff --git a/backend/tests/test_channels.py b/backend/tests/test_channels.py index 8e7546ded..aaa5997b9 100644 --- a/backend/tests/test_channels.py +++ b/backend/tests/test_channels.py @@ -7,7 +7,7 @@ import json import tempfile from pathlib import Path from types import SimpleNamespace -from unittest.mock import AsyncMock, MagicMock +from unittest.mock import AsyncMock, MagicMock, patch import pytest @@ -1988,6 +1988,47 @@ class TestSlackSendRetry: _run(go()) + +class TestSlackAllowedUsers: + def test_numeric_allowed_users_match_string_event_user_id(self): + from app.channels.slack import SlackChannel + + bus = MessageBus() + bus.publish_inbound = AsyncMock() + channel = SlackChannel( + bus=bus, + config={"allowed_users": [123456]}, + ) + channel._loop = MagicMock() + channel._loop.is_running.return_value = True + channel._add_reaction = MagicMock() + channel._send_running_reply = MagicMock() + + event = { + "user": "123456", + "text": "hello from slack", + "channel": "C123", + "ts": "1710000000.000100", + } + + def submit_coro(coro, loop): + coro.close() + return MagicMock() + + with patch( + "app.channels.slack.asyncio.run_coroutine_threadsafe", + side_effect=submit_coro, + ) as submit: + channel._handle_message_event(event) + + channel._add_reaction.assert_called_once_with("C123", "1710000000.000100", "eyes") + channel._send_running_reply.assert_called_once_with("C123", "1710000000.000100") + submit.assert_called_once() + inbound = bus.publish_inbound.call_args.args[0] + assert inbound.user_id == "123456" + assert inbound.chat_id == "C123" + assert inbound.text == "hello from slack" + def test_raises_after_all_retries_exhausted(self): from app.channels.slack import SlackChannel From ca2fb95ee6bae08073ad058ecaecbf180c32a50c Mon Sep 17 00:00:00 2001 From: greatmengqi Date: Sun, 5 Apr 2026 21:07:35 +0800 Subject: [PATCH 12/12] feat: unified serve.sh with gateway mode support (#1847) --- Makefile | 82 +++++-- README.md | 54 +++++ backend/CLAUDE.md | 25 ++- docker/docker-compose-dev.yaml | 17 +- docker/docker-compose.yaml | 15 +- docker/nginx/nginx.conf | 36 +--- docker/provisioner/Dockerfile | 2 + scripts/deploy.sh | 130 ++++++++++-- scripts/docker.sh | 45 +++- scripts/serve.sh | 376 ++++++++++++++++++++------------- scripts/start-daemon.sh | 145 +------------ 11 files changed, 551 insertions(+), 376 deletions(-) diff --git a/Makefile b/Makefile index e74a02db3..d190de3e6 100644 --- a/Makefile +++ b/Makefile @@ -1,6 +1,6 @@ # DeerFlow - Unified Development Environment -.PHONY: help config config-upgrade check install dev dev-daemon start stop up down clean docker-init docker-start docker-stop docker-logs docker-logs-frontend docker-logs-gateway +.PHONY: help config config-upgrade check install dev dev-pro dev-daemon dev-daemon-pro start start-pro start-daemon start-daemon-pro stop up up-pro down clean docker-init docker-start docker-start-pro docker-stop docker-logs docker-logs-frontend docker-logs-gateway BASH ?= bash @@ -20,18 +20,25 @@ help: @echo " make install - Install all dependencies (frontend + backend)" @echo " make setup-sandbox - Pre-pull sandbox container image (recommended)" @echo " make dev - Start all services in development mode (with hot-reloading)" - @echo " make dev-daemon - Start all services in background (daemon mode)" + @echo " make dev-pro - Start in dev + Gateway mode (experimental, no LangGraph server)" + @echo " make dev-daemon - Start dev services in background (daemon mode)" + @echo " make dev-daemon-pro - Start dev daemon + Gateway mode (experimental)" @echo " make start - Start all services in production mode (optimized, no hot-reloading)" + @echo " make start-pro - Start in prod + Gateway mode (experimental)" + @echo " make start-daemon - Start prod services in background (daemon mode)" + @echo " make start-daemon-pro - Start prod daemon + Gateway mode (experimental)" @echo " make stop - Stop all running services" @echo " make clean - Clean up processes and temporary files" @echo "" @echo "Docker Production Commands:" @echo " make up - Build and start production Docker services (localhost:2026)" + @echo " make up-pro - Build and start production Docker in Gateway mode (experimental)" @echo " make down - Stop and remove production Docker containers" @echo "" @echo "Docker Development Commands:" @echo " make docker-init - Pull the sandbox image" @echo " make docker-start - Start Docker services (mode-aware from config.yaml, localhost:2026)" + @echo " make docker-start-pro - Start Docker in Gateway mode (experimental, no LangGraph container)" @echo " make docker-stop - Stop Docker development services" @echo " make docker-logs - View Docker development logs" @echo " make docker-logs-frontend - View Docker frontend logs" @@ -105,6 +112,15 @@ else @./scripts/serve.sh --dev endif +# Start all services in dev + Gateway mode (experimental: agent runtime embedded in Gateway) +dev-pro: + @$(PYTHON) ./scripts/check.py +ifeq ($(OS),Windows_NT) + @call scripts\run-with-git-bash.cmd ./scripts/serve.sh --dev --gateway +else + @./scripts/serve.sh --dev --gateway +endif + # Start all services in production mode (with optimizations) start: @$(PYTHON) ./scripts/check.py @@ -114,30 +130,54 @@ else @./scripts/serve.sh --prod endif +# Start all services in prod + Gateway mode (experimental) +start-pro: + @$(PYTHON) ./scripts/check.py +ifeq ($(OS),Windows_NT) + @call scripts\run-with-git-bash.cmd ./scripts/serve.sh --prod --gateway +else + @./scripts/serve.sh --prod --gateway +endif + # Start all services in daemon mode (background) dev-daemon: @$(PYTHON) ./scripts/check.py ifeq ($(OS),Windows_NT) - @call scripts\run-with-git-bash.cmd ./scripts/start-daemon.sh + @call scripts\run-with-git-bash.cmd ./scripts/serve.sh --dev --daemon else - @./scripts/start-daemon.sh + @./scripts/serve.sh --dev --daemon +endif + +# Start daemon + Gateway mode (experimental) +dev-daemon-pro: + @$(PYTHON) ./scripts/check.py +ifeq ($(OS),Windows_NT) + @call scripts\run-with-git-bash.cmd ./scripts/serve.sh --dev --gateway --daemon +else + @./scripts/serve.sh --dev --gateway --daemon +endif + +# Start prod services in daemon mode (background) +start-daemon: + @$(PYTHON) ./scripts/check.py +ifeq ($(OS),Windows_NT) + @call scripts\run-with-git-bash.cmd ./scripts/serve.sh --prod --daemon +else + @./scripts/serve.sh --prod --daemon +endif + +# Start prod daemon + Gateway mode (experimental) +start-daemon-pro: + @$(PYTHON) ./scripts/check.py +ifeq ($(OS),Windows_NT) + @call scripts\run-with-git-bash.cmd ./scripts/serve.sh --prod --gateway --daemon +else + @./scripts/serve.sh --prod --gateway --daemon endif # Stop all services stop: - @echo "Stopping all services..." - @-pkill -f "langgraph dev" 2>/dev/null || true - @-pkill -f "uvicorn app.gateway.app:app" 2>/dev/null || true - @-pkill -f "next dev" 2>/dev/null || true - @-pkill -f "next start" 2>/dev/null || true - @-pkill -f "next-server" 2>/dev/null || true - @-pkill -f "next-server" 2>/dev/null || true - @-nginx -c $(PWD)/docker/nginx/nginx.local.conf -p $(PWD) -s quit 2>/dev/null || true - @sleep 1 - @-pkill -9 nginx 2>/dev/null || true - @echo "Cleaning up sandbox containers..." - @-./scripts/cleanup-containers.sh deer-flow-sandbox 2>/dev/null || true - @echo "✓ All services stopped" + @./scripts/serve.sh --stop # Clean up clean: stop @@ -159,6 +199,10 @@ docker-init: docker-start: @./scripts/docker.sh start +# Start Docker in Gateway mode (experimental) +docker-start-pro: + @./scripts/docker.sh start --gateway + # Stop Docker development environment docker-stop: @./scripts/docker.sh stop @@ -181,6 +225,10 @@ docker-logs-gateway: up: @./scripts/deploy.sh +# Build and start production services in Gateway mode +up-pro: + @./scripts/deploy.sh --gateway + # Stop and remove production containers down: @./scripts/deploy.sh down diff --git a/README.md b/README.md index 1c1f6dcdf..14aec9fc6 100644 --- a/README.md +++ b/README.md @@ -280,6 +280,60 @@ On Windows, run the local development flow from Git Bash. Native `cmd.exe` and P 6. **Access**: http://localhost:2026 +#### Startup Modes + +DeerFlow supports multiple startup modes across two dimensions: + +- **Dev / Prod** — dev enables hot-reload; prod uses pre-built frontend +- **Standard / Gateway** — standard uses a separate LangGraph server (4 processes); Gateway mode (experimental) embeds the agent runtime in the Gateway API (3 processes) + +| | **Local Foreground** | **Local Daemon** | **Docker Dev** | **Docker Prod** | +|---|---|---|---|---| +| **Dev** | `./scripts/serve.sh --dev`
`make dev` | `./scripts/serve.sh --dev --daemon`
`make dev-daemon` | `./scripts/docker.sh start`
`make docker-start` | — | +| **Dev + Gateway** | `./scripts/serve.sh --dev --gateway`
`make dev-pro` | `./scripts/serve.sh --dev --gateway --daemon`
`make dev-daemon-pro` | `./scripts/docker.sh start --gateway`
`make docker-start-pro` | — | +| **Prod** | `./scripts/serve.sh --prod`
`make start` | `./scripts/serve.sh --prod --daemon`
`make start-daemon` | — | `./scripts/deploy.sh`
`make up` | +| **Prod + Gateway** | `./scripts/serve.sh --prod --gateway`
`make start-pro` | `./scripts/serve.sh --prod --gateway --daemon`
`make start-daemon-pro` | — | `./scripts/deploy.sh --gateway`
`make up-pro` | + +| Action | Local | Docker Dev | Docker Prod | +|---|---|---|---| +| **Stop** | `./scripts/serve.sh --stop`
`make stop` | `./scripts/docker.sh stop`
`make docker-stop` | `./scripts/deploy.sh down`
`make down` | +| **Restart** | `./scripts/serve.sh --restart [flags]` | `./scripts/docker.sh restart` | — | + +> **Gateway mode** eliminates the LangGraph server process — the Gateway API handles agent execution directly via async tasks, managing its own concurrency. + +#### Why Gateway Mode? + +In standard mode, DeerFlow runs a dedicated [LangGraph Platform](https://langchain-ai.github.io/langgraph/) server alongside the Gateway API. This architecture works well but has trade-offs: + +| | Standard Mode | Gateway Mode | +|---|---|---| +| **Architecture** | Gateway (REST API) + LangGraph (agent runtime) | Gateway embeds agent runtime | +| **Concurrency** | `--n-jobs-per-worker` per worker (requires license) | `--workers` × async tasks (no per-worker cap) | +| **Containers / Processes** | 4 (frontend, gateway, langgraph, nginx) | 3 (frontend, gateway, nginx) | +| **Resource usage** | Higher (two Python runtimes) | Lower (single Python runtime) | +| **LangGraph Platform license** | Required for production images | Not required | +| **Cold start** | Slower (two services to initialize) | Faster | + +Both modes are functionally equivalent — the same agents, tools, and skills work in either mode. + +#### Docker Production Deployment + +`deploy.sh` supports building and starting separately. Images are mode-agnostic — runtime mode is selected at start time: + +```bash +# One-step (build + start) +deploy.sh # standard mode (default) +deploy.sh --gateway # gateway mode + +# Two-step (build once, start with any mode) +deploy.sh build # build all images +deploy.sh start # start in standard mode +deploy.sh start --gateway # start in gateway mode + +# Stop +deploy.sh down +``` + ### Advanced #### Sandbox Mode diff --git a/backend/CLAUDE.md b/backend/CLAUDE.md index a45b14253..846429e40 100644 --- a/backend/CLAUDE.md +++ b/backend/CLAUDE.md @@ -13,6 +13,10 @@ DeerFlow is a LangGraph-based AI super agent system with a full-stack architectu - **Nginx** (port 2026): Unified reverse proxy entry point - **Provisioner** (port 8002, optional in Docker dev): Started only when sandbox is configured for provisioner/Kubernetes mode +**Runtime Modes**: +- **Standard mode** (`make dev`): LangGraph Server handles agent execution as a separate process. 4 processes total. +- **Gateway mode** (`make dev-pro`, experimental): Agent runtime embedded in Gateway via `RunManager` + `run_agent()` + `StreamBridge` (`packages/harness/deerflow/runtime/`). Service manages its own concurrency via async tasks. 3 processes total, no LangGraph Server. + **Project Structure**: ``` deer-flow/ @@ -80,6 +84,8 @@ When making code changes, you MUST update the relevant documentation: make check # Check system requirements make install # Install all dependencies (frontend + backend) make dev # Start all services (LangGraph + Gateway + Frontend + Nginx), with config.yaml preflight +make dev-pro # Gateway mode (experimental): skip LangGraph, agent runtime embedded in Gateway +make start-pro # Production + Gateway mode (experimental) make stop # Stop all services ``` @@ -436,8 +442,25 @@ make dev This starts all services and makes the application available at `http://localhost:2026`. +**All startup modes:** + +| | **Local Foreground** | **Local Daemon** | **Docker Dev** | **Docker Prod** | +|---|---|---|---|---| +| **Dev** | `./scripts/serve.sh --dev`
`make dev` | `./scripts/serve.sh --dev --daemon`
`make dev-daemon` | `./scripts/docker.sh start`
`make docker-start` | — | +| **Dev + Gateway** | `./scripts/serve.sh --dev --gateway`
`make dev-pro` | `./scripts/serve.sh --dev --gateway --daemon`
`make dev-daemon-pro` | `./scripts/docker.sh start --gateway`
`make docker-start-pro` | — | +| **Prod** | `./scripts/serve.sh --prod`
`make start` | `./scripts/serve.sh --prod --daemon`
`make start-daemon` | — | `./scripts/deploy.sh`
`make up` | +| **Prod + Gateway** | `./scripts/serve.sh --prod --gateway`
`make start-pro` | `./scripts/serve.sh --prod --gateway --daemon`
`make start-daemon-pro` | — | `./scripts/deploy.sh --gateway`
`make up-pro` | + +| Action | Local | Docker Dev | Docker Prod | +|---|---|---|---| +| **Stop** | `./scripts/serve.sh --stop`
`make stop` | `./scripts/docker.sh stop`
`make docker-stop` | `./scripts/deploy.sh down`
`make down` | +| **Restart** | `./scripts/serve.sh --restart [flags]` | `./scripts/docker.sh restart` | — | + +Gateway mode embeds the agent runtime in Gateway, no LangGraph server. + **Nginx routing**: -- `/api/langgraph/*` → LangGraph Server (2024) +- Standard mode: `/api/langgraph/*` → LangGraph Server (2024) +- Gateway mode: `/api/langgraph/*` → Gateway embedded runtime (8001) (via envsubst) - `/api/*` (other) → Gateway API (8001) - `/` (non-API) → Frontend (3000) diff --git a/docker/docker-compose-dev.yaml b/docker/docker-compose-dev.yaml index c0749ba9d..53dcb80b2 100644 --- a/docker/docker-compose-dev.yaml +++ b/docker/docker-compose-dev.yaml @@ -19,8 +19,6 @@ services: # cluster via the K8s API. # Backend accesses sandboxes directly via host.docker.internal:{NodePort}. provisioner: - profiles: - - provisioner build: context: ./provisioner dockerfile: Dockerfile @@ -59,20 +57,25 @@ services: # ── Reverse Proxy ────────────────────────────────────────────────────── # Routes API traffic to gateway/langgraph and (optionally) provisioner. - # Select nginx config via NGINX_CONF: - # - nginx.local.conf (default): no provisioner route (local/aio modes) - # - nginx.conf: includes provisioner route (provisioner mode) + # LANGGRAPH_UPSTREAM and LANGGRAPH_REWRITE control gateway vs standard + # routing (processed by envsubst at container start). nginx: image: nginx:alpine container_name: deer-flow-nginx ports: - "2026:2026" volumes: - - ./nginx/${NGINX_CONF:-nginx.conf}:/etc/nginx/nginx.conf:ro + - ./nginx/nginx.conf:/etc/nginx/nginx.conf.template:ro + environment: + - LANGGRAPH_UPSTREAM=${LANGGRAPH_UPSTREAM:-langgraph:2024} + - LANGGRAPH_REWRITE=${LANGGRAPH_REWRITE:-/} + command: > + sh -c "envsubst '$$LANGGRAPH_UPSTREAM $$LANGGRAPH_REWRITE' + < /etc/nginx/nginx.conf.template > /etc/nginx/nginx.conf + && nginx -g 'daemon off;'" depends_on: - frontend - gateway - - langgraph networks: - deer-flow-dev restart: unless-stopped diff --git a/docker/docker-compose.yaml b/docker/docker-compose.yaml index 98d549878..8c1432da7 100644 --- a/docker/docker-compose.yaml +++ b/docker/docker-compose.yaml @@ -29,11 +29,17 @@ services: ports: - "${PORT:-2026}:2026" volumes: - - ./nginx/${NGINX_CONF:-nginx.conf}:/etc/nginx/nginx.conf:ro + - ./nginx/nginx.conf:/etc/nginx/nginx.conf.template:ro + environment: + - LANGGRAPH_UPSTREAM=${LANGGRAPH_UPSTREAM:-langgraph:2024} + - LANGGRAPH_REWRITE=${LANGGRAPH_REWRITE:-/} + command: > + sh -c "envsubst '$$LANGGRAPH_UPSTREAM $$LANGGRAPH_REWRITE' + < /etc/nginx/nginx.conf.template > /etc/nginx/nginx.conf + && nginx -g 'daemon off;'" depends_on: - frontend - gateway - - langgraph networks: - deer-flow restart: unless-stopped @@ -68,7 +74,7 @@ services: UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:0.7.20} UV_INDEX_URL: ${UV_INDEX_URL:-https://pypi.org/simple} container_name: deer-flow-gateway - command: sh -c "cd backend && PYTHONPATH=. uv run uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001 --workers 2" + command: sh -c "cd backend && PYTHONPATH=. uv run uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001 --workers ${GATEWAY_WORKERS:-4}" volumes: - ${DEER_FLOW_CONFIG_PATH}:/app/backend/config.yaml:ro - ${DEER_FLOW_EXTENSIONS_CONFIG_PATH}:/app/backend/extensions_config.json:ro @@ -160,13 +166,12 @@ services: # ── Sandbox Provisioner (optional, Kubernetes mode) ──────────────────────── provisioner: - profiles: - - provisioner build: context: ./provisioner dockerfile: Dockerfile args: APT_MIRROR: ${APT_MIRROR:-} + PIP_INDEX_URL: ${PIP_INDEX_URL:-} container_name: deer-flow-provisioner volumes: - ~/.kube/config:/root/.kube/config:ro diff --git a/docker/nginx/nginx.conf b/docker/nginx/nginx.conf index da570f709..c9a7be32b 100644 --- a/docker/nginx/nginx.conf +++ b/docker/nginx/nginx.conf @@ -27,7 +27,7 @@ http { } upstream langgraph { - server langgraph:2024; + server ${LANGGRAPH_UPSTREAM}; } upstream frontend { @@ -57,9 +57,11 @@ http { } # LangGraph API routes - # Rewrites /api/langgraph/* to /* before proxying + # In standard mode: /api/langgraph/* → langgraph:2024 (rewrite to /*) + # In gateway mode: /api/langgraph/* → gateway:8001 (rewrite to /api/*) + # Controlled by LANGGRAPH_UPSTREAM and LANGGRAPH_REWRITE env vars. location /api/langgraph/ { - rewrite ^/api/langgraph/(.*) /$1 break; + rewrite ^/api/langgraph/(.*) ${LANGGRAPH_REWRITE}$1 break; proxy_pass http://langgraph; proxy_http_version 1.1; @@ -84,34 +86,6 @@ http { chunked_transfer_encoding on; } - # Experimental: Gateway-backed LangGraph-compatible API - # Frontend can opt-in via NEXT_PUBLIC_LANGGRAPH_BASE_URL=/api/langgraph-compat - location /api/langgraph-compat/ { - rewrite ^/api/langgraph-compat/(.*) /api/$1 break; - proxy_pass http://gateway; - proxy_http_version 1.1; - - # Headers - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_set_header Connection ''; - - # SSE/Streaming support - proxy_buffering off; - proxy_cache off; - proxy_set_header X-Accel-Buffering no; - - # Timeouts for long-running requests - proxy_connect_timeout 600s; - proxy_send_timeout 600s; - proxy_read_timeout 600s; - - # Chunked transfer encoding - chunked_transfer_encoding on; - } - # Custom API: Models endpoint location /api/models { proxy_pass http://gateway; diff --git a/docker/provisioner/Dockerfile b/docker/provisioner/Dockerfile index 6f30e778d..96ef93156 100644 --- a/docker/provisioner/Dockerfile +++ b/docker/provisioner/Dockerfile @@ -1,6 +1,7 @@ FROM python:3.12-slim-bookworm ARG APT_MIRROR +ARG PIP_INDEX_URL # Optionally override apt mirror for restricted networks (e.g. APT_MIRROR=mirrors.aliyun.com) RUN if [ -n "${APT_MIRROR}" ]; then \ @@ -15,6 +16,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ # Install Python dependencies RUN pip install --no-cache-dir \ + ${PIP_INDEX_URL:+--index-url "$PIP_INDEX_URL"} \ fastapi \ "uvicorn[standard]" \ kubernetes diff --git a/scripts/deploy.sh b/scripts/deploy.sh index 66df240ef..26cb3bc29 100755 --- a/scripts/deploy.sh +++ b/scripts/deploy.sh @@ -1,16 +1,57 @@ #!/usr/bin/env bash # -# deploy.sh - Build and start (or stop) DeerFlow production services +# deploy.sh - Build, start, or stop DeerFlow production services # -# Usage: -# deploy.sh [up] — build images and start containers (default) -# deploy.sh down — stop and remove containers +# Commands: +# deploy.sh [--MODE] — build + start (default: --standard) +# deploy.sh build — build all images (mode-agnostic) +# deploy.sh start [--MODE] — start from pre-built images (default: --standard) +# deploy.sh down — stop and remove containers +# +# Runtime modes: +# --standard (default) All services including LangGraph server. +# --gateway No LangGraph container; nginx routes /api/langgraph/* +# to the Gateway compat API instead. +# +# Sandbox mode (local / aio / provisioner) is auto-detected from config.yaml. +# +# Examples: +# deploy.sh # build + start in standard mode +# deploy.sh --gateway # build + start in gateway mode +# deploy.sh build # build all images +# deploy.sh start --gateway # start pre-built images in gateway mode +# deploy.sh down # stop and remove containers # # Must be run from the repo root directory. set -e -CMD="${1:-up}" +RUNTIME_MODE="standard" + +case "${1:-}" in + build|start|down) + CMD="$1" + if [ -n "${2:-}" ]; then + case "$2" in + --standard) RUNTIME_MODE="standard" ;; + --gateway) RUNTIME_MODE="gateway" ;; + *) echo "Unknown mode: $2"; echo "Usage: deploy.sh [build|start|down] [--standard|--gateway]"; exit 1 ;; + esac + fi + ;; + --standard|--gateway) + CMD="" + RUNTIME_MODE="${1#--}" + ;; + "") + CMD="" + ;; + *) + echo "Unknown argument: $1" + echo "Usage: deploy.sh [build|start|down] [--standard|--gateway]" + exit 1 + ;; +esac REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" cd "$REPO_ROOT" @@ -150,6 +191,32 @@ if [ "$CMD" = "down" ]; then exit 0 fi +# ── build ──────────────────────────────────────────────────────────────────── +# Build produces mode-agnostic images. No --gateway or sandbox detection needed. + +if [ "$CMD" = "build" ]; then + echo "==========================================" + echo " DeerFlow — Building Images" + echo "==========================================" + echo "" + + # Docker socket is needed for compose to parse volume specs + if [ -z "$DEER_FLOW_DOCKER_SOCKET" ]; then + export DEER_FLOW_DOCKER_SOCKET="/var/run/docker.sock" + fi + + "${COMPOSE_CMD[@]}" build + + echo "" + echo "==========================================" + echo " ✓ Images built successfully" + echo "==========================================" + echo "" + echo " Next: deploy.sh start [--gateway]" + echo "" + exit 0 +fi + # ── Banner ──────────────────────────────────────────────────────────────────── echo "==========================================" @@ -157,19 +224,28 @@ echo " DeerFlow Production Deployment" echo "==========================================" echo "" -# ── Step 1: Detect sandbox mode ────────────────────────────────────────────── +# ── Detect runtime configuration ──────────────────────────────────────────── +# Only needed for start / up — determines which containers to launch. sandbox_mode="$(detect_sandbox_mode)" echo -e "${BLUE}Sandbox mode: $sandbox_mode${NC}" -if [ "$sandbox_mode" = "provisioner" ]; then - services="" - extra_args="--profile provisioner" -else - services="frontend gateway langgraph nginx" - extra_args="" -fi +echo -e "${BLUE}Runtime mode: $RUNTIME_MODE${NC}" +case "$RUNTIME_MODE" in + gateway) + export LANGGRAPH_UPSTREAM=gateway:8001 + export LANGGRAPH_REWRITE=/api/ + services="frontend gateway nginx" + ;; + standard) + services="frontend gateway langgraph nginx" + ;; +esac + +if [ "$sandbox_mode" = "provisioner" ]; then + services="$services provisioner" +fi # ── DEER_FLOW_DOCKER_SOCKET ─────────────────────────────────────────────────── @@ -189,22 +265,34 @@ fi echo "" -# ── Step 2: Build and start ─────────────────────────────────────────────────── +# ── Start / Up ─────────────────────────────────────────────────────────────── -echo "Building images and starting containers..." -echo "" - -# shellcheck disable=SC2086 -"${COMPOSE_CMD[@]}" $extra_args up --build -d --remove-orphans $services +if [ "$CMD" = "start" ]; then + echo "Starting containers (no rebuild)..." + echo "" + # shellcheck disable=SC2086 + "${COMPOSE_CMD[@]}" up -d --remove-orphans $services +else + # Default: build + start + echo "Building images and starting containers..." + echo "" + # shellcheck disable=SC2086 + "${COMPOSE_CMD[@]}" up --build -d --remove-orphans $services +fi echo "" echo "==========================================" -echo " DeerFlow is running!" +echo " DeerFlow is running! ($RUNTIME_MODE mode)" echo "==========================================" echo "" echo " 🌐 Application: http://localhost:${PORT:-2026}" echo " 📡 API Gateway: http://localhost:${PORT:-2026}/api/*" -echo " 🤖 LangGraph: http://localhost:${PORT:-2026}/api/langgraph/*" +if [ "$RUNTIME_MODE" = "gateway" ]; then + echo " 🤖 Runtime: Gateway embedded" + echo " API: /api/langgraph/* → Gateway (compat)" +else + echo " 🤖 LangGraph: http://localhost:${PORT:-2026}/api/langgraph/*" +fi echo "" echo " Manage:" echo " make down — stop and remove containers" diff --git a/scripts/docker.sh b/scripts/docker.sh index bc20d4177..0ef1896fe 100755 --- a/scripts/docker.sh +++ b/scripts/docker.sh @@ -148,9 +148,18 @@ init() { } # Start Docker development environment +# Usage: start [--gateway] start() { local sandbox_mode local services + local gateway_mode=false + + # Check for --gateway flag + for arg in "$@"; do + if [ "$arg" = "--gateway" ]; then + gateway_mode=true + fi + done echo "==========================================" echo " Starting DeerFlow Docker Development" @@ -159,12 +168,21 @@ start() { sandbox_mode="$(detect_sandbox_mode)" - if [ "$sandbox_mode" = "provisioner" ]; then - services="frontend gateway langgraph provisioner nginx" + if $gateway_mode; then + services="frontend gateway nginx" + if [ "$sandbox_mode" = "provisioner" ]; then + services="frontend gateway provisioner nginx" + fi else services="frontend gateway langgraph nginx" + if [ "$sandbox_mode" = "provisioner" ]; then + services="frontend gateway langgraph provisioner nginx" + fi fi + if $gateway_mode; then + echo -e "${BLUE}Runtime: Gateway mode (experimental) — no LangGraph container${NC}" + fi echo -e "${BLUE}Detected sandbox mode: $sandbox_mode${NC}" if [ "$sandbox_mode" = "provisioner" ]; then echo -e "${BLUE}Provisioner enabled (Kubernetes mode).${NC}" @@ -213,6 +231,12 @@ start() { fi fi + # Set nginx routing for gateway mode (envsubst in nginx container) + if $gateway_mode; then + export LANGGRAPH_UPSTREAM=gateway:8001 + export LANGGRAPH_REWRITE=/api/ + fi + echo "Building and starting containers..." cd "$DOCKER_DIR" && $COMPOSE_CMD up --build -d --remove-orphans $services echo "" @@ -222,7 +246,12 @@ start() { echo "" echo " 🌐 Application: http://localhost:2026" echo " 📡 API Gateway: http://localhost:2026/api/*" - echo " 🤖 LangGraph: http://localhost:2026/api/langgraph/*" + if $gateway_mode; then + echo " 🤖 Runtime: Gateway embedded" + echo " API: /api/langgraph/* → Gateway (compat)" + else + echo " 🤖 LangGraph: http://localhost:2026/api/langgraph/*" + fi echo "" echo " 📋 View logs: make docker-logs" echo " 🛑 Stop: make docker-stop" @@ -300,9 +329,10 @@ help() { echo "Usage: $0 [options]" echo "" echo "Commands:" - echo " init - Pull the sandbox image (speeds up first Pod startup)" - echo " start - Start Docker services (auto-detects sandbox mode from config.yaml)" - echo " restart - Restart all running Docker services" + echo " init - Pull the sandbox image (speeds up first Pod startup)" + echo " start - Start Docker services (auto-detects sandbox mode from config.yaml)" + echo " start --gateway - Start without LangGraph container (Gateway mode, experimental)" + echo " restart - Restart all running Docker services" echo " logs [option] - View Docker development logs" echo " --frontend View frontend logs only" echo " --gateway View gateway logs only" @@ -320,7 +350,8 @@ main() { init ;; start) - start + shift + start "$@" ;; restart) restart diff --git a/scripts/serve.sh b/scripts/serve.sh index fe1556a91..bd810e05e 100755 --- a/scripts/serve.sh +++ b/scripts/serve.sh @@ -1,15 +1,40 @@ #!/usr/bin/env bash # -# start.sh - Start all DeerFlow development services +# serve.sh — Unified DeerFlow service launcher +# +# Usage: +# ./scripts/serve.sh [--dev|--prod] [--gateway] [--daemon] [--stop|--restart] +# +# Modes: +# --dev Development mode with hot-reload (default) +# --prod Production mode, pre-built frontend, no hot-reload +# --gateway Gateway mode (experimental): skip LangGraph server, +# agent runtime embedded in Gateway API +# --daemon Run all services in background (nohup), exit after startup +# +# Actions: +# --skip-install Skip dependency installation (faster restart) +# --stop Stop all running services and exit +# --restart Stop all services, then start with the given mode flags +# +# Examples: +# ./scripts/serve.sh --dev # Standard dev (4 processes) +# ./scripts/serve.sh --dev --gateway # Gateway dev (3 processes) +# ./scripts/serve.sh --prod --gateway # Gateway prod (3 processes) +# ./scripts/serve.sh --dev --daemon # Standard dev, background +# ./scripts/serve.sh --dev --gateway --daemon # Gateway dev, background +# ./scripts/serve.sh --stop # Stop all services +# ./scripts/serve.sh --restart --dev --gateway # Restart in gateway mode # # Must be run from the repo root directory. set -e -REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +REPO_ROOT="$(builtin cd "$(dirname "${BASH_SOURCE[0]}")/.." >/dev/null 2>&1 && pwd -P)" cd "$REPO_ROOT" -# ── Load environment variables from .env ────────────────────────────────────── +# ── Load .env ──────────────────────────────────────────────────────────────── + if [ -f "$REPO_ROOT/.env" ]; then set -a source "$REPO_ROOT/.env" @@ -19,14 +44,80 @@ fi # ── Argument parsing ───────────────────────────────────────────────────────── DEV_MODE=true +GATEWAY_MODE=false +DAEMON_MODE=false +SKIP_INSTALL=false +ACTION="start" # start | stop | restart + for arg in "$@"; do case "$arg" in - --dev) DEV_MODE=true ;; - --prod) DEV_MODE=false ;; - *) echo "Unknown argument: $arg"; echo "Usage: $0 [--dev|--prod]"; exit 1 ;; + --dev) DEV_MODE=true ;; + --prod) DEV_MODE=false ;; + --gateway) GATEWAY_MODE=true ;; + --daemon) DAEMON_MODE=true ;; + --skip-install) SKIP_INSTALL=true ;; + --stop) ACTION="stop" ;; + --restart) ACTION="restart" ;; + *) + echo "Unknown argument: $arg" + echo "Usage: $0 [--dev|--prod] [--gateway] [--daemon] [--skip-install] [--stop|--restart]" + exit 1 + ;; esac done +# ── Stop helper ────────────────────────────────────────────────────────────── + +stop_all() { + echo "Stopping all services..." + pkill -f "langgraph dev" 2>/dev/null || true + pkill -f "uvicorn app.gateway.app:app" 2>/dev/null || true + pkill -f "next dev" 2>/dev/null || true + pkill -f "next start" 2>/dev/null || true + pkill -f "next-server" 2>/dev/null || true + nginx -c "$REPO_ROOT/docker/nginx/nginx.local.conf" -p "$REPO_ROOT" -s quit 2>/dev/null || true + sleep 1 + pkill -9 nginx 2>/dev/null || true + ./scripts/cleanup-containers.sh deer-flow-sandbox 2>/dev/null || true + echo "✓ All services stopped" +} + +# ── Action routing ─────────────────────────────────────────────────────────── + +if [ "$ACTION" = "stop" ]; then + stop_all + exit 0 +fi + +ALREADY_STOPPED=false +if [ "$ACTION" = "restart" ]; then + stop_all + sleep 1 + ALREADY_STOPPED=true +fi + +# ── Derive runtime flags ──────────────────────────────────────────────────── + +if $GATEWAY_MODE; then + export SKIP_LANGGRAPH_SERVER=1 +fi + +# Mode label for banner +if $DEV_MODE && $GATEWAY_MODE; then + MODE_LABEL="DEV + GATEWAY (experimental)" +elif $DEV_MODE; then + MODE_LABEL="DEV (hot-reload enabled)" +elif $GATEWAY_MODE; then + MODE_LABEL="PROD + GATEWAY (experimental)" +else + MODE_LABEL="PROD (optimized)" +fi + +if $DAEMON_MODE; then + MODE_LABEL="$MODE_LABEL [daemon]" +fi + +# Frontend command if $DEV_MODE; then FRONTEND_CMD="pnpm run dev" else @@ -35,46 +126,26 @@ else elif command -v python >/dev/null 2>&1; then PYTHON_BIN="python" else - echo "Python is required to generate BETTER_AUTH_SECRET, but neither python3 nor python was found." + echo "Python is required to generate BETTER_AUTH_SECRET." exit 1 fi FRONTEND_CMD="env BETTER_AUTH_SECRET=$($PYTHON_BIN -c 'import secrets; print(secrets.token_hex(16))') pnpm run preview" fi -# ── Stop existing services ──────────────────────────────────────────────────── - -echo "Stopping existing services if any..." -pkill -f "langgraph dev" 2>/dev/null || true -pkill -f "uvicorn app.gateway.app:app" 2>/dev/null || true -pkill -f "next dev" 2>/dev/null || true -pkill -f "next-server" 2>/dev/null || true -nginx -c "$REPO_ROOT/docker/nginx/nginx.local.conf" -p "$REPO_ROOT" -s quit 2>/dev/null || true -sleep 1 -pkill -9 nginx 2>/dev/null || true -killall -9 nginx 2>/dev/null || true -./scripts/cleanup-containers.sh deer-flow-sandbox 2>/dev/null || true -sleep 1 - -# ── Banner ──────────────────────────────────────────────────────────────────── - -echo "" -echo "==========================================" -echo " Starting DeerFlow Development Server" -echo "==========================================" -echo "" -if $DEV_MODE; then - echo " Mode: DEV (hot-reload enabled)" - echo " Tip: run \`make start\` in production mode" +# Extra flags for uvicorn/langgraph +LANGGRAPH_EXTRA_FLAGS="--no-reload" +if $DEV_MODE && ! $DAEMON_MODE; then + GATEWAY_EXTRA_FLAGS="--reload --reload-include='*.yaml' --reload-include='.env' --reload-exclude='*.pyc' --reload-exclude='__pycache__' --reload-exclude='sandbox/' --reload-exclude='.deer-flow/'" else - echo " Mode: PROD (hot-reload disabled)" - echo " Tip: run \`make dev\` to start in development mode" + GATEWAY_EXTRA_FLAGS="" +fi + +# ── Stop existing services (skip if restart already did it) ────────────────── + +if ! $ALREADY_STOPPED; then + stop_all + sleep 1 fi -echo "" -echo "Services starting up..." -echo " → Backend: LangGraph + Gateway" -echo " → Frontend: Next.js" -echo " → Nginx: Reverse Proxy" -echo "" # ── Config check ───────────────────────────────────────────────────────────── @@ -84,64 +155,108 @@ if ! { \ [ -f config.yaml ]; \ }; then echo "✗ No DeerFlow config file found." - echo " Checked these locations:" - echo " - $DEER_FLOW_CONFIG_PATH (when DEER_FLOW_CONFIG_PATH is set)" - echo " - backend/config.yaml" - echo " - ./config.yaml" - echo "" - echo " Run 'make config' from the repo root to generate ./config.yaml, then set required model API keys in .env or your config file." + echo " Run 'make config' to generate config.yaml." exit 1 fi -# ── Auto-upgrade config ────────────────────────────────────────────────── - "$REPO_ROOT/scripts/config-upgrade.sh" -# ── Cleanup trap ───────────────────────────────────────────────────────────── +# ── Install dependencies ──────────────────────────────────────────────────── + +if ! $SKIP_INSTALL; then + echo "Syncing dependencies..." + (cd backend && uv sync --quiet) || { echo "✗ Backend dependency install failed"; exit 1; } + (cd frontend && pnpm install --silent) || { echo "✗ Frontend dependency install failed"; exit 1; } + echo "✓ Dependencies synced" +else + echo "⏩ Skipping dependency install (--skip-install)" +fi + +# ── Sync frontend .env.local ───────────────────────────────────────────────── +# Next.js .env.local takes precedence over process env vars. +# The script manages the NEXT_PUBLIC_LANGGRAPH_BASE_URL line to ensure +# the frontend routes match the active backend mode. + +FRONTEND_ENV_LOCAL="$REPO_ROOT/frontend/.env.local" +ENV_KEY="NEXT_PUBLIC_LANGGRAPH_BASE_URL" + +sync_frontend_env() { + if $GATEWAY_MODE; then + # Point frontend to Gateway's compat API + if [ -f "$FRONTEND_ENV_LOCAL" ] && grep -q "^${ENV_KEY}=" "$FRONTEND_ENV_LOCAL"; then + sed -i.bak "s|^${ENV_KEY}=.*|${ENV_KEY}=/api/langgraph-compat|" "$FRONTEND_ENV_LOCAL" && rm -f "${FRONTEND_ENV_LOCAL}.bak" + else + echo "${ENV_KEY}=/api/langgraph-compat" >> "$FRONTEND_ENV_LOCAL" + fi + else + # Remove override — frontend falls back to /api/langgraph (standard) + if [ -f "$FRONTEND_ENV_LOCAL" ] && grep -q "^${ENV_KEY}=" "$FRONTEND_ENV_LOCAL"; then + sed -i.bak "/^${ENV_KEY}=/d" "$FRONTEND_ENV_LOCAL" && rm -f "${FRONTEND_ENV_LOCAL}.bak" + fi + fi +} + +sync_frontend_env + +# ── Banner ─────────────────────────────────────────────────────────────────── + +echo "" +echo "==========================================" +echo " Starting DeerFlow" +echo "==========================================" +echo "" +echo " Mode: $MODE_LABEL" +echo "" +echo " Services:" +if ! $GATEWAY_MODE; then + echo " LangGraph → localhost:2024 (agent runtime)" +fi +echo " Gateway → localhost:8001 (REST API$(if $GATEWAY_MODE; then echo " + agent runtime"; fi))" +echo " Frontend → localhost:3000 (Next.js)" +echo " Nginx → localhost:2026 (reverse proxy)" +echo "" + +# ── Cleanup handler ────────────────────────────────────────────────────────── cleanup() { trap - INT TERM echo "" - echo "Shutting down services..." - if [ "${SKIP_LANGGRAPH_SERVER:-0}" != "1" ]; then - pkill -f "langgraph dev" 2>/dev/null || true - fi - pkill -f "uvicorn app.gateway.app:app" 2>/dev/null || true - pkill -f "next dev" 2>/dev/null || true - pkill -f "next start" 2>/dev/null || true - pkill -f "next-server" 2>/dev/null || true - # Kill nginx using the captured PID first (most reliable), - # then fall back to pkill/killall for any stray nginx workers. - if [ -n "${NGINX_PID:-}" ] && kill -0 "$NGINX_PID" 2>/dev/null; then - kill -TERM "$NGINX_PID" 2>/dev/null || true - sleep 1 - kill -9 "$NGINX_PID" 2>/dev/null || true - fi - pkill -9 nginx 2>/dev/null || true - killall -9 nginx 2>/dev/null || true - echo "Cleaning up sandbox containers..." - ./scripts/cleanup-containers.sh deer-flow-sandbox 2>/dev/null || true - echo "✓ All services stopped" + stop_all exit 0 } + trap cleanup INT TERM -# ── Start services ──────────────────────────────────────────────────────────── +# ── Helper: start a service ────────────────────────────────────────────────── + +# run_service NAME COMMAND PORT TIMEOUT +# In daemon mode, wraps with nohup. Waits for port to be ready. +run_service() { + local name="$1" cmd="$2" port="$3" timeout="$4" + + echo "Starting $name..." + if $DAEMON_MODE; then + nohup sh -c "$cmd" > /dev/null 2>&1 & + else + sh -c "$cmd" & + fi + + ./scripts/wait-for-port.sh "$port" "$timeout" "$name" || { + local logfile="logs/$(echo "$name" | tr '[:upper:]' '[:lower:]' | tr ' ' '-').log" + echo "✗ $name failed to start." + [ -f "$logfile" ] && tail -20 "$logfile" + cleanup + } + echo "✓ $name started on localhost:$port" +} + +# ── Start services ─────────────────────────────────────────────────────────── mkdir -p logs mkdir -p temp/client_body_temp temp/proxy_temp temp/fastcgi_temp temp/uwsgi_temp temp/scgi_temp -if $DEV_MODE; then - LANGGRAPH_EXTRA_FLAGS="--no-reload" - GATEWAY_EXTRA_FLAGS="--reload --reload-include='*.yaml' --reload-include='.env' --reload-exclude='*.pyc' --reload-exclude='__pycache__' --reload-exclude='sandbox/' --reload-exclude='.deer-flow/'" -else - LANGGRAPH_EXTRA_FLAGS="--no-reload" - GATEWAY_EXTRA_FLAGS="" -fi - -if [ "${SKIP_LANGGRAPH_SERVER:-0}" != "1" ]; then - echo "Starting LangGraph server..." - # Read log_level from config.yaml, fallback to env var, then to "info" +# 1. LangGraph (skip in gateway mode) +if ! $GATEWAY_MODE; then CONFIG_LOG_LEVEL=$(grep -m1 '^log_level:' config.yaml 2>/dev/null | awk '{print $2}' | tr -d ' ') LANGGRAPH_LOG_LEVEL="${LANGGRAPH_LOG_LEVEL:-${CONFIG_LOG_LEVEL:-info}}" LANGGRAPH_JOBS_PER_WORKER="${LANGGRAPH_JOBS_PER_WORKER:-10}" @@ -150,85 +265,54 @@ if [ "${SKIP_LANGGRAPH_SERVER:-0}" != "1" ]; then if [ "$LANGGRAPH_ALLOW_BLOCKING" = "1" ]; then LANGGRAPH_ALLOW_BLOCKING_FLAG="--allow-blocking" fi - (cd backend && NO_COLOR=1 uv run langgraph dev --no-browser $LANGGRAPH_ALLOW_BLOCKING_FLAG --n-jobs-per-worker "$LANGGRAPH_JOBS_PER_WORKER" --server-log-level $LANGGRAPH_LOG_LEVEL $LANGGRAPH_EXTRA_FLAGS > ../logs/langgraph.log 2>&1) & - ./scripts/wait-for-port.sh 2024 60 "LangGraph" || { - echo " See logs/langgraph.log for details" - tail -20 logs/langgraph.log - if grep -qE "config_version|outdated|Environment variable .* not found|KeyError|ValidationError|config\.yaml" logs/langgraph.log 2>/dev/null; then - echo "" - echo " Hint: This may be a configuration issue. Try running 'make config-upgrade' to update your config.yaml." - fi - cleanup - } - echo "✓ LangGraph server started on localhost:2024" + run_service "LangGraph" \ + "cd backend && NO_COLOR=1 uv run langgraph dev --no-browser $LANGGRAPH_ALLOW_BLOCKING_FLAG --n-jobs-per-worker $LANGGRAPH_JOBS_PER_WORKER --server-log-level $LANGGRAPH_LOG_LEVEL $LANGGRAPH_EXTRA_FLAGS > ../logs/langgraph.log 2>&1" \ + 2024 60 else - echo "⏩ Skipping LangGraph server (SKIP_LANGGRAPH_SERVER=1)" - echo " Use /api/langgraph-compat/* via Gateway instead" + echo "⏩ Skipping LangGraph (Gateway mode — runtime embedded in Gateway)" fi -echo "Starting Gateway API..." -(cd backend && PYTHONPATH=. uv run uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001 $GATEWAY_EXTRA_FLAGS > ../logs/gateway.log 2>&1) & -./scripts/wait-for-port.sh 8001 30 "Gateway API" || { - echo "✗ Gateway API failed to start. Last log output:" - tail -60 logs/gateway.log - echo "" - echo "Likely configuration errors:" - grep -E "Failed to load configuration|Environment variable .* not found|config\.yaml.*not found" logs/gateway.log | tail -5 || true - echo "" - echo " Hint: Try running 'make config-upgrade' to update your config.yaml with the latest fields." - cleanup -} -echo "✓ Gateway API started on localhost:8001" +# 2. Gateway API +run_service "Gateway" \ + "cd backend && PYTHONPATH=. uv run uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001 $GATEWAY_EXTRA_FLAGS > ../logs/gateway.log 2>&1" \ + 8001 30 -echo "Starting Frontend..." -(cd frontend && $FRONTEND_CMD > ../logs/frontend.log 2>&1) & -./scripts/wait-for-port.sh 3000 120 "Frontend" || { - echo " See logs/frontend.log for details" - tail -20 logs/frontend.log - cleanup -} -echo "✓ Frontend started on localhost:3000" +# 3. Frontend +run_service "Frontend" \ + "cd frontend && $FRONTEND_CMD > ../logs/frontend.log 2>&1" \ + 3000 120 -echo "Starting Nginx reverse proxy..." -nginx -g 'daemon off;' -c "$REPO_ROOT/docker/nginx/nginx.local.conf" -p "$REPO_ROOT" > logs/nginx.log 2>&1 & -NGINX_PID=$! -./scripts/wait-for-port.sh 2026 10 "Nginx" || { - echo " See logs/nginx.log for details" - tail -10 logs/nginx.log - cleanup -} -echo "✓ Nginx started on localhost:2026" +# 4. Nginx +run_service "Nginx" \ + "nginx -g 'daemon off;' -c '$REPO_ROOT/docker/nginx/nginx.local.conf' -p '$REPO_ROOT' > logs/nginx.log 2>&1" \ + 2026 10 -# ── Ready ───────────────────────────────────────────────────────────────────── +# ── Ready ──────────────────────────────────────────────────────────────────── echo "" echo "==========================================" -if $DEV_MODE; then - echo " ✓ DeerFlow development server is running!" -else - echo " ✓ DeerFlow production server is running!" -fi +echo " ✓ DeerFlow is running! [$MODE_LABEL]" echo "==========================================" echo "" -echo " 🌐 Application: http://localhost:2026" -echo " 📡 API Gateway: http://localhost:2026/api/*" -if [ "${SKIP_LANGGRAPH_SERVER:-0}" = "1" ]; then - echo " 🤖 LangGraph: skipped (SKIP_LANGGRAPH_SERVER=1)" +echo " 🌐 http://localhost:2026" +echo "" +if $GATEWAY_MODE; then + echo " Routing: Frontend → Nginx → Gateway (embedded runtime)" + echo " API: /api/langgraph-compat/* → Gateway agent runtime" else - echo " 🤖 LangGraph: http://localhost:2026/api/langgraph/* (served by langgraph dev)" -fi -echo " 🧪 LangGraph Compat (experimental): http://localhost:2026/api/langgraph-compat/* (served by Gateway)" -if [ "${SKIP_LANGGRAPH_SERVER:-0}" = "1" ]; then - echo "" - echo " 💡 Set NEXT_PUBLIC_LANGGRAPH_BASE_URL=/api/langgraph-compat in frontend/.env.local" + echo " Routing: Frontend → Nginx → LangGraph + Gateway" + echo " API: /api/langgraph/* → LangGraph server (2024)" fi +echo " /api/* → Gateway REST API (8001)" echo "" -echo " 📋 Logs:" -echo " - LangGraph: logs/langgraph.log" -echo " - Gateway: logs/gateway.log" -echo " - Frontend: logs/frontend.log" -echo " - Nginx: logs/nginx.log" +echo " 📋 Logs: logs/{langgraph,gateway,frontend,nginx}.log" echo "" -echo "Press Ctrl+C to stop all services" -wait +if $DAEMON_MODE; then + echo " 🛑 Stop: make stop" + # Detach — trap is no longer needed + trap - INT TERM +else + echo " Press Ctrl+C to stop all services" + wait +fi diff --git a/scripts/start-daemon.sh b/scripts/start-daemon.sh index cce8c65bb..8822b73a0 100755 --- a/scripts/start-daemon.sh +++ b/scripts/start-daemon.sh @@ -1,146 +1,9 @@ #!/usr/bin/env bash # -# start-daemon.sh - Start all DeerFlow development services in daemon mode +# start-daemon.sh — Start DeerFlow in daemon (background) mode # -# This script starts DeerFlow services in the background without keeping -# the terminal connection. Logs are written to separate files. -# -# Must be run from the repo root directory. - -set -e +# Thin wrapper around serve.sh --daemon. +# Kept for backward compatibility. REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" -cd "$REPO_ROOT" - -# ── Stop existing services ──────────────────────────────────────────────────── - -echo "Stopping existing services if any..." -pkill -f "langgraph dev" 2>/dev/null || true -pkill -f "uvicorn app.gateway.app:app" 2>/dev/null || true -pkill -f "next dev" 2>/dev/null || true -nginx -c "$REPO_ROOT/docker/nginx/nginx.local.conf" -p "$REPO_ROOT" -s quit 2>/dev/null || true -sleep 1 -pkill -9 nginx 2>/dev/null || true -./scripts/cleanup-containers.sh deer-flow-sandbox 2>/dev/null || true -sleep 1 - -# ── Banner ──────────────────────────────────────────────────────────────────── - -echo "" -echo "==========================================" -echo " Starting DeerFlow in Daemon Mode" -echo "==========================================" -echo "" - -# ── Config check ───────────────────────────────────────────────────────────── - -if ! { \ - [ -n "$DEER_FLOW_CONFIG_PATH" ] && [ -f "$DEER_FLOW_CONFIG_PATH" ] || \ - [ -f backend/config.yaml ] || \ - [ -f config.yaml ]; \ - }; then - echo "✗ No DeerFlow config file found." - echo " Checked these locations:" - echo " - $DEER_FLOW_CONFIG_PATH (when DEER_FLOW_CONFIG_PATH is set)" - echo " - backend/config.yaml" - echo " - ./config.yaml" - echo "" - echo " Run 'make config' from the repo root to generate ./config.yaml, then set required model API keys in .env or your config file." - exit 1 -fi - -# ── Auto-upgrade config ────────────────────────────────────────────────── - -"$REPO_ROOT/scripts/config-upgrade.sh" - -# ── Cleanup on failure ─────────────────────────────────────────────────────── - -cleanup_on_failure() { - echo "Failed to start services, cleaning up..." - pkill -f "langgraph dev" 2>/dev/null || true - pkill -f "uvicorn app.gateway.app:app" 2>/dev/null || true - pkill -f "next dev" 2>/dev/null || true - nginx -c "$REPO_ROOT/docker/nginx/nginx.local.conf" -p "$REPO_ROOT" -s quit 2>/dev/null || true - sleep 1 - pkill -9 nginx 2>/dev/null || true - echo "✓ Cleanup complete" -} - -trap cleanup_on_failure INT TERM - -# ── Start services ──────────────────────────────────────────────────────────── - -mkdir -p logs -mkdir -p temp/client_body_temp temp/proxy_temp temp/fastcgi_temp temp/uwsgi_temp temp/scgi_temp - -echo "Starting LangGraph server..." -LANGGRAPH_JOBS_PER_WORKER="${LANGGRAPH_JOBS_PER_WORKER:-10}" -LANGGRAPH_ALLOW_BLOCKING="${LANGGRAPH_ALLOW_BLOCKING:-0}" -LANGGRAPH_ALLOW_BLOCKING_FLAG="" -if [ "$LANGGRAPH_ALLOW_BLOCKING" = "1" ]; then - LANGGRAPH_ALLOW_BLOCKING_FLAG="--allow-blocking" -fi -nohup sh -c "cd backend && NO_COLOR=1 uv run langgraph dev --no-browser ${LANGGRAPH_ALLOW_BLOCKING_FLAG} --no-reload --n-jobs-per-worker ${LANGGRAPH_JOBS_PER_WORKER} > ../logs/langgraph.log 2>&1" & -./scripts/wait-for-port.sh 2024 60 "LangGraph" || { - echo "✗ LangGraph failed to start. Last log output:" - tail -60 logs/langgraph.log - if grep -qE "config_version|outdated|Environment variable .* not found|KeyError|ValidationError|config\.yaml" logs/langgraph.log 2>/dev/null; then - echo "" - echo " Hint: This may be a configuration issue. Try running 'make config-upgrade' to update your config.yaml." - fi - cleanup_on_failure - exit 1 -} -echo "✓ LangGraph server started on localhost:2024" - -echo "Starting Gateway API..." -nohup sh -c 'cd backend && PYTHONPATH=. uv run uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001 > ../logs/gateway.log 2>&1' & -./scripts/wait-for-port.sh 8001 30 "Gateway API" || { - echo "✗ Gateway API failed to start. Last log output:" - tail -60 logs/gateway.log - echo "" - echo " Hint: Try running 'make config-upgrade' to update your config.yaml with the latest fields." - cleanup_on_failure - exit 1 -} -echo "✓ Gateway API started on localhost:8001" - -echo "Starting Frontend..." -nohup sh -c 'cd frontend && pnpm run dev > ../logs/frontend.log 2>&1' & -./scripts/wait-for-port.sh 3000 120 "Frontend" || { - echo "✗ Frontend failed to start. Last log output:" - tail -60 logs/frontend.log - cleanup_on_failure - exit 1 -} -echo "✓ Frontend started on localhost:3000" - -echo "Starting Nginx reverse proxy..." -nohup sh -c 'nginx -g "daemon off;" -c "$1/docker/nginx/nginx.local.conf" -p "$1" > logs/nginx.log 2>&1' _ "$REPO_ROOT" & -./scripts/wait-for-port.sh 2026 10 "Nginx" || { - echo "✗ Nginx failed to start. Last log output:" - tail -60 logs/nginx.log - cleanup_on_failure - exit 1 -} -echo "✓ Nginx started on localhost:2026" - -# ── Ready ───────────────────────────────────────────────────────────────────── - -echo "" -echo "==========================================" -echo " DeerFlow is running in daemon mode!" -echo "==========================================" -echo "" -echo " 🌐 Application: http://localhost:2026" -echo " 📡 API Gateway: http://localhost:2026/api/*" -echo " 🤖 LangGraph: http://localhost:2026/api/langgraph/*" -echo "" -echo " 📋 Logs:" -echo " - LangGraph: logs/langgraph.log" -echo " - Gateway: logs/gateway.log" -echo " - Frontend: logs/frontend.log" -echo " - Nginx: logs/nginx.log" -echo "" -echo " 🛑 Stop daemon: make stop" -echo "" +exec "$REPO_ROOT/scripts/serve.sh" --dev --daemon "$@"