github/ChatDev
1
0
Fork 0
mirror of https://github.com/OpenBMB/ChatDev.git synced 2026-04-25 11:18:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

initial commit of chatdev 2.0

Browse Source
This commit is contained in:
NA-Wen 2026-01-07 16:24:01 +08:00
commit f0db945ed3
566 changed files with 61882 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.env Executable file
View File

@ -0,0 +1,4 @@
BASE_URL=
API_KEY=
SERPER_DEV_API_KEY=
JINA_API_KEY=

2
.gitattributes vendored Executable file
View File

@ -0,0 +1,2 @@
# Auto detect text files and perform LF normalization
* text=auto

18
.gitignore vendored Normal file
View File

@ -0,0 +1,18 @@
*.pyc
.DS_Store
.idea
.vscode
__pycache__/
.env/
.venv/
env/
venv/
.idea
.venv
.uv-cache
logs
node_modules
frontend/.vscode
WareHouse/
data/
temp/

201
LICENSE Normal file
View File

@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2025 OpenBMB
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

305
README-zh.md Executable file
View File

@ -0,0 +1,305 @@
# ChatDev 2.0 - DevAll
<p align="center">
<img src="frontend/public/media/logo.png" alt="DevAll Logo" width="500"/>
</p>
<p align="center">
<strong>用于开发一切的零代码多智能体平台</strong>
</p>
<p align="center">
【<a href="./README.md">English</a> | <a href="./README-zh.md">简体中文</a>】
</p>
<p align="center">
【📚 <a href="#开发者">开发者</a> | 👥 <a href="#主要贡献者">贡献者</a>｜⭐️ <a href="https://github.com/OpenBMB/ChatDev/tree/legacy">ChatDev 1.0 (Legacy)</a>】
</p>
## 📖 概览
ChatDev 已从一个专门的软件开发多智能体系统演变为一个全面的多智能体编排平台。
- <a href="https://github.com/OpenBMB/ChatDev/tree/main">**ChatDev 2.0 (DevAll)**</a> 是一个用于“开发一切”的**零代码多智能体平台**。它通过简单的配置，赋能用户快速构建并执行定制化的多智能体系统。无需编写代码，用户即可定义智能体、工作流和任务，以编排如数据可视化、3D 生成和深度调研等复杂场景。
- <a href="https://github.com/OpenBMB/ChatDev/tree/chatdev1.0">**ChatDev 1.0 (经典版)**</a> 以**虚拟软件公司**模式运行。它通过各种智能体（如 CEO、CTO、程序员）参与专门的功能研讨会，实现整个软件开发生命周期的自动化——包括设计、编码、测试和文档编写。它是沟通型智能体协作的基石范式。
## 🎉 新闻
• **2026年1月7日：🚀 我们非常高兴地宣布 ChatDev 2.0 (DevAll) 正式发布！** 该版本引入了全新的零代码多智能体编排平台。经典的 ChatDev (v1.x) 已移至 [`chatdev1.0`](https://github.com/OpenBMB/ChatDev/tree/chatdev1.0) 分支进行维护。
<details>
<summary>历史新闻</summary>
•2025年9月24日：🎉 我们的论文 [Multi-Agent Collaboration via Evolving Orchestration](https://arxiv.org/abs/2505.19591) 已被 NeurIPS 2025 接收。其实现可在本仓库的 `puppeteer` 分支中找到。
•2025年5月26日：🎉 我们提出了一种新型的“木偶戏”式范式，用于大语言模型智能体之间的多智能体协作。通过利用强化学习优化的可学习中央编排器，我们的方法动态地激活并排列智能体，以构建高效、情境感知的推理路径。这种方法不仅提高了推理质量，还降低了计算成本，使多智能体协作在复杂任务中具有可扩展性和适应性。详见论文：[Multi-Agent Collaboration via Evolving Orchestration](https://arxiv.org/abs/2505.19591)。
<p align="center">
<img src='./assets/puppeteer.png' width=800>
</p>
•2024年6月25日：🎉 为了促进 LLM 驱动的多智能体协作🤖🤖及相关领域的发展，ChatDev 团队策划了一系列开创性的论文📄，并以[开源](https://github.com/OpenBMB/ChatDev/tree/main/MultiAgentEbook)交互式电子书📚的形式呈现。现在您可以在 [电子书网站](https://thinkwee.top/multiagent_ebook) 探索最新进展，并下载 [论文列表](https://github.com/OpenBMB/ChatDev/blob/main/MultiAgentEbook/papers.csv)。
<p align="center">
<img src='./assets/ebook.png' width=800>
</p>
•2024年6月12日：我们推出了多智能体协作网络 (MacNet) 🎉，它利用有向无环图 (DAG) 通过语言交互促进智能体之间有效的面向任务的协作 🤖🤖。MacNet 支持跨各种拓扑结构以及在超过一千个智能体之间进行协作，且不超出上下文限制。MacNet 更加通用和可扩展，可以被视为 ChatDev 链式拓扑的更高级版本。我们的预印本论文可在 [https://arxiv.org/abs/2406.07155](https://arxiv.org/abs/2406.07155) 获取。该技术已整合到 [macnet](https://github.com/OpenBMB/ChatDev/tree/macnet) 分支，增强了对多样化组织结构的支持，并提供了除软件开发之外的更丰富解决方案（例如，逻辑推理、数据分析、故事生成等）。
<p align="center">
<img src='./assets/macnet.png' width=500>
</p>
• 2024年5月7日，我们推出了“迭代经验提炼”（IER），这是一种新方法，指导者智能体和助手智能体通过增强捷径导向的经验来高效适应新任务。这种方法涵盖了在一系列任务中获取、利用、传播和消除经验的过程，使过程更加简短高效。我们的预印本论文可在 https://arxiv.org/abs/2405.04219 获取，该技术将很快整合到 ChatDev 中。
<p align="center">
<img src='./assets/ier.png' width=220>
</p>
• 2024年1月25日：我们已在 ChatDev 中集成了体验式共同学习模块。请参阅 [体验式共同学习指南](wiki.md#co-tracking)。
• 2023年12月28日：我们提出了体验式共同学习，这是一种创新方法，指导者智能体和助手智能体积累捷径导向的经验，以有效地解决新任务，减少重复错误并提高效率。请查看我们的预印本论文 https://arxiv.org/abs/2312.17025，该技术将很快集成到 ChatDev 中。
<p align="center">
<img src='./assets/ecl.png' width=860>
</p>
• 2023年11月15日：我们推出了 ChatDev SaaS 平台，使软件开发人员和创新创业者能够以极低的成本高效构建软件，并消除准入门槛。请访问 https://chatdev.modelbest.cn/ 试用。
<p align="center">
<img src='./assets/saas.png' width=560>
</p>
• 2023年11月2日：ChatDev 现在支持一项新功能：增量开发，允许智能体在现有代码基础上进行开发。尝试 ```--config "incremental" --path "[source_code_directory_path]"``` 开始使用。
<p align="center">
<img src='./assets/increment.png' width=700>
</p>
• 2023年10月26日：ChatDev 现在支持 Docker 安全运行（感谢 [ManindraDeMel](https://github.com/ManindraDeMel) 的贡献）。请参阅 [Docker 快速开始指南](wiki.md#docker-start)。
<p align="center">
<img src='./assets/docker.png' width=400>
</p>
• 2023年9月25日：**Git** 模式现已上线，允许程序员 <img src='visualizer/static/figures/programmer.png' height=20> 利用 Git 进行版本控制。要启用此功能，只需在 ``ChatChainConfig.json`` 中将 ``"git_management"`` 设置为 ``"True"``。参见 [指南](wiki.md#git-mode)。
<p align="center">
<img src='./assets/github.png' width=600>
</p>
• 2023年9月20日：**人机交互**模式现已上线！您可以通过扮演评论员的角色 <img src='visualizer/static/figures/reviewer.png' height=20> 并向程序员 <img src='visualizer/static/figures/programmer.png' height=20> 提出建议来参与到 ChatDev 团队中；
尝试 ``python3 run.py --task [description_of_your_idea] --config "Human"``。参见 [指南](wiki.md#human-agent-interaction) 和 [示例](WareHouse/Gomoku_HumanAgentInteraction_20230920135038)。
<p align="center">
<img src='./assets/Human_intro.png' width=600>
</p>
• 2023年9月1日：**艺术**模式现已上线！您可以激活设计师智能体 <img src='visualizer/static/figures/designer.png' height=20> 来生成软件中使用的图像；
尝试 ``python3 run.py --task [description_of_your_idea] --config "Art"``。参见 [指南](wiki.md#art) 和 [示例](WareHouse/gomokugameArtExample_THUNLP_20230831122822)。
• 2023年8月28日：系统公开发布。
• 2023年8月17日：v1.0.0 版本准备发布。
• 2023年7月30日：用户可以自定义 ChatChain、Phase 和 Role 设置。此外，现在已支持在线日志模式和回放模式。
• 2023年7月16日：该项目相关的 [预印本论文](https://arxiv.org/abs/2307.07924) 发表。
• 2023年6月30日：ChatDev 仓库的初始版本发布。
</details>
## 🚀 快速开始
### 📋 环境要求
* **操作系统**: macOS / Linux / WSL / Windows
* **Python**: 3.12+
* **Node.js**: 18+
* **包管理器**: [uv](https://docs.astral.sh/uv/)
### 📦 安装
1. **后端依赖**（由 `uv` 管理 Python）：
```bash
uv sync
```
2. **前端依赖**（Vite + Vue 3）：
```bash
cd frontend && npm install
```
### ⚡️ 运行应用
1. **启动后端**：
```bash
# 从项目根目录运行
uv run python server_main.py --port 6400 --reload
```
2. **启动前端**：
```bash
cd frontend
VITE_API_BASE_URL=http://localhost:6400 npm run dev
```
> 然后访问 Web 控制台：**[http://localhost:5173](http://localhost:5173)**。
### 🔑 配置
* **环境变量**：在项目根目录创建一个 `.env` 文件。
* **模型密钥**：在 `.env` 中设置 `API_KEY` 和 `BASE_URL` 对应您的 LLM 提供商。
* **YAML 占位符**：在配置文件中使用 `${VAR}`（如 `${API_KEY}`）来引用这些变量。
---
## 💡 如何使用
### 🖥️ Web 控制台
DevAll 界面为构建和执行提供了无缝体验：
* **教程 (Tutorial)**：平台内置了全面的分步指南和文档，帮助您快速上手。
<img src="assets/tutorial-en.png"/>
* **工作流 (Workflow)**：可视化画布，用于设计您的多智能体系统。通过轻松的拖拽来配置节点参数、定义上下文流并编排复杂的智能体交互。
<img src="assets/workflow.gif"/>
* **运行 (Launch)**：启动工作流、监控实时日志、检查中间产物，并提供人机协同反馈。
<img src="assets/launch.gif"/>
### 🧰 Python SDK
对于自动化和批量处理，使用我们轻量级的 Python SDK 编排任务并直接获取结果。
```python
from runtime.sdk import run_workflow
# 执行工作流并获取最后一条节点消息
result = run_workflow(
yaml_file="yaml_instance/demo.yaml",
task_prompt="用一句话总结附件文档。",
attachments=["/path/to/document.pdf"],
variables={"API_KEY": "sk-xxxx"} # 如果需要，可覆盖 .env 中的变量
)
if result.final_message:
print(f"Output: {result.final_message.text_content()}")
```
---
<a id="开发者"></a>
## ⚙️ 给开发者
**如果您打算进行二次开发和扩展，请参阅本章节。**
您可以通过扩展节点、Provider 与工具来增强 DevAll。
项目采用模块化结构：
* **核心系统**：`server/` 承载 FastAPI 后端，`runtime/` 负责智能体抽象与工具执行。
* **编排层**：`workflow/` 负责多智能体逻辑，配置位于 `entity/`。
* **前端**：`frontend/` 是 Vue 3 Web 控制台。
* **可扩展性**：`functions/` 用于自定义 Python 工具。
相关参考文档：
* **快速开始**：[Start Guide](./docs/user_guide/zh/index.md)
* **核心模块**：[Workflow Authoring](./docs/user_guide/zh/workflow_authoring.md)、[Memory](./docs/user_guide/zh/modules/memory.md) 和 [Tooling](./docs/user_guide/zh/modules/tooling/index.md)
---
## 🌟 推荐工作流
我们为常见场景提供了开箱即用的强大模板。所有可运行的工作流配置均位于 `yaml_instance/` 目录下。
* **示例 (Demos)**：以 `demo_*.yaml` 命名的文件展示了特定功能或模块。
* **实现 (Implementations)**：直接命名的文件（如 `ChatDev_v1.yaml`）是完整的自研或复刻流程。如下所示：
### 📋 工作流合集
| 类别 | 工作流 | 案例 |
| :--- | :--- | :--- |
| **📈 数据可视化** | `data_visualization_basic.yaml`<br>`data_visualization_enhanced.yaml` | <img src="assets/cases/data_analysis/data_analysis.gif" width="100%"><br>提示词：*"Create 4–6 high-quality PNG charts for my large real-estate transactions dataset."* |
| **🛠️ 3D 场景生成**<br>*(需要 [Blender](https://www.blender.org/) 和 [blender-mcp](https://github.com/ahujasid/blender-mcp))* | `blender_3d_builder_simple.yaml`<br>`blender_3d_builder_hub.yaml`<br>`blender_scientific_illustration.yaml` | <img src="assets/cases/3d_generation/3d.gif" width="100%"><br>提示词：*"Please build a Christmas tree."* |
| **🎮 游戏开发** | `GameDev_v1.yaml`<br>`ChatDev_v1.yaml` | <img src="assets/cases/game_development/game.gif" width="100%"><br>提示词：*"Please help me design and develop a Tank Battle game."* |
| **📚 深度研究** | `deep_research_v1.yaml` | <img src="assets/cases/deep_research/deep_research.gif" width="85%"><br>提示词：*"Research about recent advances in the field of LLM-based agent RL"* |
| **🎓 教学视频** | `teach_video.yaml` | <img src="assets/cases/video_generation/video.gif" width="140%"><br>提示词：*"讲一下什么是凸优化"* |
------
### 💡 使用指南
对于这些实现，您可以使用 **Launch** 标签页来执行它们。
1. **选择**：在 **Launch** 标签页选择一个工作流。
2. **上传**：如果需要，上传相关文件（例如用于数据分析的 `.csv`）。
3. **提示**：输入您的请求（例如*“可视化销售趋势”*或*“设计一个贪吃蛇游戏”*）。
---
## 🤝 参与贡献
我们欢迎社区的任何形式的贡献！无论是修复 Bug、添加新的工作流模板，还是分享由 DevAll 生成的优质案例/产物，您的帮助都至关重要。欢迎通过提交 **Issue** 或 **Pull Request** 来参与。
通过参与贡献，您的名字将被列入下方的 **贡献者** 名单中。请查看 [开发者指南](#开发者) 开始您的贡献之旅！
### 👥 贡献者
#### 主要贡献者
<table>
<tr>
<td align="center"><a href="https://github.com/NA-Wen"><img src="https://github.com/NA-Wen.png?size=100" width="64px;" alt=""/><br /><sub><b>NA-Wen</b></sub></a></td>
<td align="center"><a href="https://github.com/zxrys"><img src="https://github.com/zxrys.png?size=100" width="64px;" alt=""/><br /><sub><b>zxrys</b></sub></a></td>
<td align="center"><a href="https://github.com/swugi"><img src="https://github.com/swugi.png?size=100" width="64px;" alt=""/><br /><sub><b>swugi</b></sub></a></td>
<td align="center"><a href="https://github.com/huatl98"><img src="https://github.com/huatl98.png?size=100" width="64px;" alt=""/><br /><sub><b>huatl98</b></sub></a></td>
</tr>
</table>
#### 贡献者
<table>
<tr>
<td align="center"><a href="https://github.com/shiowen"><img src="https://github.com/shiowen.png?size=100" width="64px;" alt=""/><br /><sub><b>shiowen</b></sub></a></td>
<td align="center"><a href="https://github.com/kilo2127"><img src="https://github.com/kilo2127.png?size=100" width="64px;" alt=""/><br /><sub><b>kilo2127</b></sub></a></td>
<td align="center"><a href="https://github.com/AckerlyLau"><img src="https://github.com/AckerlyLau.png?size=100" width="64px;" alt=""/><br /><sub><b>AckerlyLau</b></sub></a></td>
</table>
## 🤝 致谢
<a href="http://nlp.csai.tsinghua.edu.cn/"><img src="assets/thunlp.png" height=50pt></a>&nbsp;&nbsp;
<a href="https://modelbest.cn/"><img src="assets/modelbest.png" height=50pt></a>&nbsp;&nbsp;
<a href="https://github.com/OpenBMB/AgentVerse/"><img src="assets/agentverse.png" height=50pt></a>&nbsp;&nbsp;
<a href="https://github.com/OpenBMB/RepoAgent"><img src="assets/repoagent.png" height=50pt></a>
<a href="https://app.commanddash.io/agent?github=https://github.com/OpenBMB/ChatDev"><img src="assets/CommandDash.png" height=50pt></a>
<a href="www.teachmaster.cn"><img src="assets/teachmaster.png" height=50pt></a>
<a href="https://github.com/OpenBMB/AppCopilot"><img src="assets/appcopilot.png" height=50pt></a>
## 🔎 引用
```
@article{chatdev,
title = {ChatDev: Communicative Agents for Software Development},
author = {Chen Qian and Wei Liu and Hongzhang Liu and Nuo Chen and Yufan Dang and Jiahao Li and Cheng Yang and Weize Chen and Yusheng Su and Xin Cong and Juyuan Xu and Dahai Li and Zhiyuan Liu and Maosong Sun},
journal = {arXiv preprint arXiv:2307.07924},
url = {https://arxiv.org/abs/2307.07924},
year = {2023}
}
@article{colearning,
title = {Experiential Co-Learning of Software-Developing Agents},
author = {Chen Qian and Yufan Dang and Jiahao Li and Wei Liu and Zihao Xie and Yifei Wang and Weize Chen and Cheng Yang and Xin Cong and Xiaoyin Che and Zhiyuan Liu and Maosong Sun},
journal = {arXiv preprint arXiv:2312.17025},
url = {https://arxiv.org/abs/2312.17025},
year = {2023}
}
@article{macnet,
title={Scaling Large-Language-Model-based Multi-Agent Collaboration},
author={Chen Qian and Zihao Xie and Yifei Wang and Wei Liu and Yufan Dang and Zhuoyun Du and Weize Chen and Cheng Yang and Zhiyuan Liu and Maosong Sun}
journal={arXiv preprint arXiv:2406.07155},
url = {https://arxiv.org/abs/2406.07155},
year={2024}
}
@article{iagents,
title={Autonomous Agents for Collaborative Task under Information Asymmetry},
author={Wei Liu and Chenxi Wang and Yifei Wang and Zihao Xie and Rennai Qiu and Yufan Dnag and Zhuoyun Du and Weize Chen and Cheng Yang and Chen Qian},
journal={arXiv preprint arXiv:2406.14928},
url = {https://arxiv.org/abs/2406.14928},
year={2024}
}
@article{puppeteer,
title={Multi-Agent Collaboration via Evolving Orchestration},
author={Yufan Dang and Chen Qian and Xueheng Luo and Jingru Fan and Zihao Xie and Ruijie Shi and Weize Chen and Cheng Yang and Xiaoyin Che and Ye Tian and Xuantang Xiong and Lei Han and Zhiyuan Liu and Maosong Sun},
journal={arXiv preprint arXiv:2505.19591},
url={https://arxiv.org/abs/2505.19591},
year={2025}
}
```
## 📬 联系方式
如果您有任何问题、反馈或希望取得联系，请随时通过电子邮件发送至 [qianc62@gmail.com](mailto:qianc62@gmail.com)

307
README.md Normal file
View File

@ -0,0 +1,307 @@
# ChatDev 2.0 - DevAll
<p align="center">
<img src="frontend/public/media/logo.png" alt="DevAll Logo" width="500"/>
</p>
<p align="center">
<strong>A Zero-Code Multi-Agent Platform for Developing Everything</strong>
</p>
<p align="center">
【<a href="./README.md">English</a> | <a href="./README-zh.md">简体中文</a>】
</p>
<p align="center">
【📚 <a href="#developers">Developers</a> | 👥 <a href="#primary-contributors">Contributors</a>｜⭐️ <a href="https://github.com/OpenBMB/ChatDev/tree/legacy">ChatDev 1.0 (Legacy)</a>】
</p>
## 📖 Overview
ChatDev has evolved from a specialized software development multi-agent system into a comprehensive multi-agent orchestration platform.
- <a href="https://github.com/OpenBMB/ChatDev/tree/main">**ChatDev 2.0 (DevAll)**</a> is a **Zero-Code Multi-Agent Platform** for "Developing Everything". It empowers users to rapidly build and execute customized multi-agent systems through simple configuration. No coding is required—users can define agents, workflows, and tasks to orchestrate complex scenarios such as data visualization, 3D generation, and deep research.
- <a href="https://github.com/OpenBMB/ChatDev/tree/chatdev1.0">**ChatDev 1.0 (Legacy)**</a> operates as a **Virtual Software Company**. It utilizes various intelligent agents (e.g., CEO, CTO, Programmer) participating in specialized functional seminars to automate the entire software development life cycle—including designing, coding, testing, and documenting. It serves as the foundational paradigm for communicative agent collaboration.
## 🎉 News
• **Jan 07, 2026: 🚀 We are excited to announce the official release of ChatDev 2.0 (DevAll)!** This version introduces a zero-code multi-agent orchestration platform. The classic ChatDev (v1.x) has been moved to the [`chatdev1.0`](https://github.com/OpenBMB/ChatDev/tree/chatdev1.0) branch for maintenance.
<details>
<summary>Old News</summary>
•Sep 24, 2025: 🎉 Our paper [Multi-Agent Collaboration via Evolving Orchestration](https://arxiv.org/abs/2505.19591) has been accepted to NeurIPS 2025. The implementation is available in the `puppeteer` branch of this repository.
•May 26, 2025: 🎉 We propose a novel puppeteer-style paradigm for multi-agent collaboration among large language model based agents. By leveraging a learnable central orchestrator optimized with reinforcement learning, our method dynamically activates and sequences agents to construct efficient, context-aware reasoning paths. This approach not only improves reasoning quality but also reduces computational costs, enabling scalable and adaptable multi-agent cooperation in complex tasks.
See our paper in [Multi-Agent Collaboration via Evolving Orchestration](https://arxiv.org/abs/2505.19591).
<p align="center">
<img src='./assets/puppeteer.png' width=800>
</p>
•June 25, 2024: 🎉To foster development in LLM-powered multi-agent collaboration🤖🤖 and related fields, the ChatDev team has curated a collection of seminal papers📄 presented in a [open-source](https://github.com/OpenBMB/ChatDev/tree/main/MultiAgentEbook) interactive e-book📚 format. Now you can explore the latest advancements on the [Ebook Website](https://thinkwee.top/multiagent_ebook) and download the [paper list](https://github.com/OpenBMB/ChatDev/blob/main/MultiAgentEbook/papers.csv).
<p align="center">
<img src='./assets/ebook.png' width=800>
</p>
•June 12, 2024: We introduced Multi-Agent Collaboration Networks (MacNet) 🎉, which utilize directed acyclic graphs to facilitate effective task-oriented collaboration among agents through linguistic interactions 🤖🤖. MacNet supports co-operation across various topologies and among more than a thousand agents without exceeding context limits. More versatile and scalable, MacNet can be considered as a more advanced version of ChatDev's chain-shaped topology. Our preprint paper is available at [https://arxiv.org/abs/2406.07155](https://arxiv.org/abs/2406.07155). This technique has been incorporated into the [macnet](https://github.com/OpenBMB/ChatDev/tree/macnet) branch, enhancing support for diverse organizational structures and offering richer solutions beyond software development (e.g., logical reasoning, data analysis, story generation, and more).
<p align="center">
<img src='./assets/macnet.png' width=500>
</p>
• May 07, 2024, we introduced "Iterative Experience Refinement" (IER), a novel method where instructor and assistant agents enhance shortcut-oriented experiences to efficiently adapt to new tasks. This approach encompasses experience acquisition, utilization, propagation and elimination across a series of tasks and making the pricess shorter and efficient. Our preprint paper is available at https://arxiv.org/abs/2405.04219, and this technique will soon be incorporated into ChatDev.
<p align="center">
<img src='./assets/ier.png' width=220>
</p>
• January 25, 2024: We have integrated Experiential Co-Learning Module into ChatDev. Please see the [Experiential Co-Learning Guide](wiki.md#co-tracking).
• December 28, 2023: We present Experiential Co-Learning, an innovative approach where instructor and assistant agents accumulate shortcut-oriented experiences to effectively solve new tasks, reducing repetitive errors and enhancing efficiency. Check out our preprint paper at https://arxiv.org/abs/2312.17025 and this technique will soon be integrated into ChatDev.
<p align="center">
<img src='./assets/ecl.png' width=860>
</p>
• November 15, 2023: We launched ChatDev as a SaaS platform that enables software developers and innovative entrepreneurs to build software efficiently at a very low cost and remove the barrier to entry. Try it out at https://chatdev.modelbest.cn/.
<p align="center">
<img src='./assets/saas.png' width=560>
</p>
• November 2, 2023: ChatDev is now supported with a new feature: incremental development, which allows agents to develop upon existing codes. Try ```--config "incremental" --path "[source_code_directory_path]"``` to start it.
<p align="center">
<img src='./assets/increment.png' width=700>
</p>
• October 26, 2023: ChatDev is now supported with Docker for safe execution (thanks to contribution from [ManindraDeMel](https://github.com/ManindraDeMel)). Please see [Docker Start Guide](wiki.md#docker-start).
<p align="center">
<img src='./assets/docker.png' width=400>
</p>
• September 25, 2023: The **Git** mode is now available, enabling the programmer <img src='visualizer/static/figures/programmer.png' height=20> to utilize Git for version control. To enable this feature, simply set ``"git_management"`` to ``"True"`` in ``ChatChainConfig.json``. See [guide](wiki.md#git-mode).
<p align="center">
<img src='./assets/github.png' width=600>
</p>
• September 20, 2023: The **Human-Agent-Interaction** mode is now available! You can get involved with the ChatDev team by playing the role of reviewer <img src='visualizer/static/figures/reviewer.png' height=20> and making suggestions to the programmer <img src='visualizer/static/figures/programmer.png' height=20>;
try ``python3 run.py --task [description_of_your_idea] --config "Human"``. See [guide](wiki.md#human-agent-interaction) and [example](WareHouse/Gomoku_HumanAgentInteraction_20230920135038).
<p align="center">
<img src='./assets/Human_intro.png' width=600>
</p>
• September 1, 2023: The **Art** mode is available now! You can activate the designer agent <img src='visualizer/static/figures/designer.png' height=20> to generate images used in the software;
try ``python3 run.py --task [description_of_your_idea] --config "Art"``. See [guide](wiki.md#art) and [example](WareHouse/gomokugameArtExample_THUNLP_20230831122822).
• August 28, 2023: The system is publicly available.
• August 17, 2023: The v1.0.0 version was ready for release.
• July 30, 2023: Users can customize ChatChain, Phasea and Role settings. Additionally, both online Log mode and replay
mode are now supported.
• July 16, 2023: The [preprint paper](https://arxiv.org/abs/2307.07924) associated with this project was published.
• June 30, 2023: The initial version of the ChatDev repository was released.
</details>
## 🚀 Quick Start
### 📋 Prerequisites
* **OS**: macOS / Linux / WSL / Windows
* **Python**: 3.12+
* **Node.js**: 18+
* **Package Manager**: [uv](https://docs.astral.sh/uv/)
### 📦 Installation
1. **Backend Dependencies** (Python managed by `uv`):
```bash
uv sync
```
2. **Frontend Dependencies** (Vite + Vue 3):
```bash
cd frontend && npm install
```
### ⚡️ Run the Application
1. **Start Backend** :
```bash
# Run from the project root
uv run python server_main.py --port 6400 --reload
```
2. **Start Frontend**:
```bash
cd frontend
VITE_API_BASE_URL=http://localhost:6400 npm run dev
```
> Then access the Web Console at **[http://localhost:5173](http://localhost:5173)**.
### 🔑 Configuration
* **Environment Variables**: Create a `.env` file in the project root.
* **Model Keys**: Set `API_KEY` and `BASE_URL` in `.env` for your LLM provider.
* **YAML placeholders**: Use `${VAR}`（e.g., `${API_KEY}`）in configuration files to reference these variables.
---
## 💡 How to Use
### 🖥️ Web Console
The DevAll interface provides a seamless experience for both construction and execution
* **Tutorial**: Comprehensive step-by-step guides and documentation integrated directly into the platform to help you get started quickly.
<img src="assets/tutorial-en.png"/>
* **Workflow**: A visual canvas to design your multi-agent systems. Configure node parameters, define context flows, and orchestrate complex agent interactions with drag-and-drop ease.
<img src="assets/workflow.gif"/>
* **Launch**: Initiate workflows, monitor real-time logs, inspect intermediate artifacts, and provide human-in-the-loop feedback.
<img src="assets/launch.gif"/>
### 🧰 Python SDK
For automation and batch processing, use our lightweight Python SDK to execute workflows programmatically and retrieve results directly.
```python
from runtime.sdk import run_workflow
# Execute a workflow and get the final node message
result = run_workflow(
yaml_file="yaml_instance/demo.yaml",
task_prompt="Summarize the attached document in one sentence.",
attachments=["/path/to/document.pdf"],
variables={"API_KEY": "sk-xxxx"} # Override .env variables if needed
)
if result.final_message:
print(f"Output: {result.final_message.text_content()}")
```
---
<a id="developers"></a>
## ⚙️ For Developers
**For secondary development and extensions, please proceed with this section.**
Extend DevAll with new nodes, providers, and tools.
The project is organized into a modular structure:
* **Core Systems**: `server/` hosts the FastAPI backend, while `runtime/` manages agent abstraction and tool execution.
* **Orchestration**: `workflow/` handles the multi-agent logic, driven by configurations in `entity/`.
* **Frontend**: `frontend/` contains the Vue 3 Web Console.
* **Extensibility**: `functions/` is the place for custom Python tools.
Relevant reference documentation:
* **Getting Started**: [Start Guide](./docs/user_guide/en/index.md)
* **Core Modules**: [Workflow Authoring](./docs/user_guide/en/workflow_authoring.md), [Memory](./docs/user_guide/en/modules/memory.md), and [Tooling](./docs/user_guide/en/modules/tooling/index.md)
---
## 🌟 Featured Workflows
We provide robust, out-of-the-box templates for common scenarios. All runnable workflow configs are located in `yaml_instance/`.
* **Demos**: Files named `demo_*.yaml` showcase specific features or modules.
* **Implementations**: Files named directly (e.g., `ChatDev_v1.yaml`) are full in-house or recreated workflows. As follows:
### 📋 Workflow Collection
| Category | Workflow | Case |
| :--- | :--- | :--- |
| **📈 Data Visualization** | `data_visualization_basic.yaml`<br>`data_visualization_enhanced.yaml` | <img src="assets/cases/data_analysis/data_analysis.gif" width="100%"><br>Prompt: *"Create 4–6 high-quality PNG charts for my large real-estate transactions dataset."* |
| **🛠️ 3D Generation**<br>*(Requires [Blender](https://www.blender.org/) & [blender-mcp](https://github.com/ahujasid/blender-mcp))* | `blender_3d_builder_simple.yaml`<br>`blender_3d_builder_hub.yaml`<br>`blender_scientific_illustration.yaml` | <img src="assets/cases/3d_generation/3d.gif" width="100%"><br>Prompt: *"Please build a Christmas tree."* |
| **🎮 Game Dev** | `GameDev_v1.yaml`<br>`ChatDev_v1.yaml` | <img src="assets/cases/game_development/game.gif" width="100%"><br>Prompt: *"Please help me design and develop a Tank Battle game."* |
| **📚 Deep Research** | `deep_research_v1.yaml` | <img src="assets/cases/deep_research/deep_research.gif" width="85%"><br>Prompt: *"Research about recent advances in the field of LLM-based agent RL"* |
| **🎓 Teach Video** | `teach_video.yaml` | <img src="assets/cases/video_generation/video.gif" width="140%"><br>Prompt: *"讲一下什么是凸优化"* |
---
### 💡 Usage Guide
For those implementations, you can use the **Launch** tab to execute them.
1. **Select**: Choose a workflow in the **Launch** tab.
2. **Upload**: Upload necessary files (e.g., `.csv` for data analysis) if required.
3. **Prompt**: Enter your request (e.g., *"Visualize the sales trends"* or *"Design a snake game"*).
---
## 🤝 Contributing
We welcome contributions from the community! Whether you're fixing bugs, adding new workflow templates, or sharing high-quality cases/artifacts produced by DevAll, your help is much appreciated. Feel free to contribute by submitting **Issues** or **Pull Requests**.
By contributing to DevAll, you'll be recognized in our **Contributors** list below. Check out our [Developer Guide](#developers) to get started!
### 👥 Contributors
#### Primary Contributors
<table>
<tr>
<td align="center"><a href="https://github.com/NA-Wen"><img src="https://github.com/NA-Wen.png?size=100" width="64px;" alt=""/><br /><sub><b>NA-Wen</b></sub></a></td>
<td align="center"><a href="https://github.com/zxrys"><img src="https://github.com/zxrys.png?size=100" width="64px;" alt=""/><br /><sub><b>zxrys</b></sub></a></td>
<td align="center"><a href="https://github.com/swugi"><img src="https://github.com/swugi.png?size=100" width="64px;" alt=""/><br /><sub><b>swugi</b></sub></a></td>
<td align="center"><a href="https://github.com/huatl98"><img src="https://github.com/huatl98.png?size=100" width="64px;" alt=""/><br /><sub><b>huatl98</b></sub></a></td>
</tr>
</table>
#### Contributors
<table>
<tr>
<td align="center"><a href="https://github.com/shiowen"><img src="https://github.com/shiowen.png?size=100" width="64px;" alt=""/><br /><sub><b>shiowen</b></sub></a></td>
<td align="center"><a href="https://github.com/kilo2127"><img src="https://github.com/kilo2127.png?size=100" width="64px;" alt=""/><br /><sub><b>kilo2127</b></sub></a></td>
<td align="center"><a href="https://github.com/AckerlyLau"><img src="https://github.com/AckerlyLau.png?size=100" width="64px;" alt=""/><br /><sub><b>AckerlyLau</b></sub></a></td>
</table>
## 🤝 Acknowledgments
<a href="http://nlp.csai.tsinghua.edu.cn/"><img src="assets/thunlp.png" height=50pt></a>&nbsp;&nbsp;
<a href="https://modelbest.cn/"><img src="assets/modelbest.png" height=50pt></a>&nbsp;&nbsp;
<a href="https://github.com/OpenBMB/AgentVerse/"><img src="assets/agentverse.png" height=50pt></a>&nbsp;&nbsp;
<a href="https://github.com/OpenBMB/RepoAgent"><img src="assets/repoagent.png" height=50pt></a>
<a href="https://app.commanddash.io/agent?github=https://github.com/OpenBMB/ChatDev"><img src="assets/CommandDash.png" height=50pt></a>
<a href="www.teachmaster.cn"><img src="assets/teachmaster.png" height=50pt></a>
<a href="https://github.com/OpenBMB/AppCopilot"><img src="assets/appcopilot.png" height=50pt></a>
## 🔎 Citation
```
@article{chatdev,
title = {ChatDev: Communicative Agents for Software Development},
author = {Chen Qian and Wei Liu and Hongzhang Liu and Nuo Chen and Yufan Dang and Jiahao Li and Cheng Yang and Weize Chen and Yusheng Su and Xin Cong and Juyuan Xu and Dahai Li and Zhiyuan Liu and Maosong Sun},
journal = {arXiv preprint arXiv:2307.07924},
url = {https://arxiv.org/abs/2307.07924},
year = {2023}
}
@article{colearning,
title = {Experiential Co-Learning of Software-Developing Agents},
author = {Chen Qian and Yufan Dang and Jiahao Li and Wei Liu and Zihao Xie and Yifei Wang and Weize Chen and Cheng Yang and Xin Cong and Xiaoyin Che and Zhiyuan Liu and Maosong Sun},
journal = {arXiv preprint arXiv:2312.17025},
url = {https://arxiv.org/abs/2312.17025},
year = {2023}
}
@article{macnet,
title={Scaling Large-Language-Model-based Multi-Agent Collaboration},
author={Chen Qian and Zihao Xie and Yifei Wang and Wei Liu and Yufan Dang and Zhuoyun Du and Weize Chen and Cheng Yang and Zhiyuan Liu and Maosong Sun}
journal={arXiv preprint arXiv:2406.07155},
url = {https://arxiv.org/abs/2406.07155},
year={2024}
}
@article{iagents,
title={Autonomous Agents for Collaborative Task under Information Asymmetry},
author={Wei Liu and Chenxi Wang and Yifei Wang and Zihao Xie and Rennai Qiu and Yufan Dnag and Zhuoyun Du and Weize Chen and Cheng Yang and Chen Qian},
journal={arXiv preprint arXiv:2406.14928},
url = {https://arxiv.org/abs/2406.14928},
year={2024}
}
@article{puppeteer,
title={Multi-Agent Collaboration via Evolving Orchestration},
author={Yufan Dang and Chen Qian and Xueheng Luo and Jingru Fan and Zihao Xie and Ruijie Shi and Weize Chen and Cheng Yang and Xiaoyin Che and Ye Tian and Xuantang Xiong and Lei Han and Zhiyuan Liu and Maosong Sun},
journal={arXiv preprint arXiv:2505.19591},
url={https://arxiv.org/abs/2505.19591},
year={2025}
}
```
## 📬 Contact
If you have any questions, feedback, or would like to get in touch, please feel free to reach out to us via email at [qianc62@gmail.com](mailto:qianc62@gmail.com)

BIN
assets/CommandDash.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 123 KiB

BIN
assets/Human_intro.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 315 KiB

BIN
assets/agentverse.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.4 MiB

BIN
assets/appcopilot.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 518 KiB

BIN
assets/cases/3d_generation/3d.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 16 MiB

BIN
assets/cases/data_analysis/data_analysis.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 5.6 MiB

BIN
assets/cases/deep_research/deep_research.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.4 MiB

BIN
assets/cases/game_development/game.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 14 MiB

BIN
assets/cases/video_generation/video.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 15 MiB

BIN
assets/docker.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 243 KiB

BIN
assets/ebook.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.3 MiB

BIN
assets/ecl.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 642 KiB

BIN
assets/github.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 66 KiB

BIN
assets/ier.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 198 KiB

BIN
assets/increment.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 305 KiB

BIN
assets/intro.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.4 MiB

BIN
assets/launch.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.1 MiB

BIN
assets/macnet.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.4 MiB

BIN
assets/modelbest.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 9.1 KiB

BIN
assets/puppeteer.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 375 KiB

BIN
assets/repoagent.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 325 KiB

BIN
assets/saas.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 934 KiB

BIN
assets/teachmaster.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 326 KiB

BIN
assets/thunlp.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 52 KiB

BIN
assets/tutorial-en.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.7 MiB

BIN
assets/workflow.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.0 MiB

0
check/__init__.py Executable file
View File

121
check/check.py Executable file
View File

@ -0,0 +1,121 @@
"""Utilities for loading, validating design_0.4.0 workflows."""
from pathlib import Path
from typing import Any, Dict, Optional
from runtime.bootstrap.schema import ensure_schema_registry_populated
from check.check_yaml import validate_design
from check.check_workflow import check_workflow_structure
from entity.config_loader import prepare_design_mapping
from entity.configs import DesignConfig, ConfigError
from schema_registry import iter_node_schemas
from utils.io_utils import read_yaml
ensure_schema_registry_populated()
class DesignError(RuntimeError):
"""Raised when a workflow design cannot be loaded or validated."""
def _allowed_node_types() -> set[str]:
names = set(iter_node_schemas().keys())
if not names:
raise DesignError("No node types registered; cannot validate workflow")
return names
def _ensure_supported(graph: Dict[str, Any]) -> None:
"""Ensure the MVP constraints are satisfied for the provided graph."""
for node in graph.get("nodes", []) or []:
nid = node.get("id")
ntype = node.get("type")
allowed = _allowed_node_types()
if ntype not in allowed:
raise DesignError(
f"Unsupported node type '{ntype}' for node '{nid}'. Only {allowed} nodes are supported."
)
if ntype == "agent":
agent_cfg = node.get("config") or {}
if not isinstance(agent_cfg, dict):
raise DesignError(f"Agent node '{nid}' config must be an object")
for legacy_key in ["memory"]:
if legacy_key in agent_cfg:
raise DesignError(
f"'{legacy_key}' is deprecated. Use the new graph-level memory stores for node '{nid}'."
)
def load_config(
config_path: Path,
*,
fn_module: Optional[str] = None,
set_defaults: bool = True,
vars_override: Optional[Dict[str, Any]] = None,
) -> DesignConfig:
"""Load, validate, and sanity-check a workflow file."""
try:
raw_data = read_yaml(config_path)
except FileNotFoundError as exc:
raise DesignError(f"Design file not found: {config_path}") from exc
if not isinstance(raw_data, dict):
raise DesignError("YAML root must be a mapping")
if vars_override:
merged_vars = dict(raw_data.get("vars") or {})
merged_vars.update(vars_override)
raw_data = dict(raw_data)
raw_data["vars"] = merged_vars
data = prepare_design_mapping(raw_data, source=str(config_path))
schema_errors = validate_design(data, set_defaults=set_defaults, fn_module_ref=fn_module)
if schema_errors:
formatted = "\n".join(f"- {err}" for err in schema_errors)
raise DesignError(f"Design validation failed for '{config_path}':\n{formatted}")
try:
design = DesignConfig.from_dict(data, path="root")
except ConfigError as exc:
raise DesignError(f"Design parsing failed for '{config_path}': {exc}") from exc
logic_errors = check_workflow_structure(data)
if logic_errors:
formatted = "\n".join(f"- {err}" for err in logic_errors)
raise DesignError(f"Workflow logical issues detected for '{config_path}':\n{formatted}")
else:
print("Workflow OK.")
graph = data.get("graph") or {}
_ensure_supported(graph)
return design
def check_config(yaml_content: Any) -> str:
if not isinstance(yaml_content, dict):
return "YAML root must be a mapping"
# Skip placeholder resolution during save - users may configure env vars at runtime
# Use yaml_content directly instead of prepare_design_mapping()
schema_errors = validate_design(yaml_content)
if schema_errors:
formatted = "\n".join(f"- {err}" for err in schema_errors)
return formatted
logic_errors = check_workflow_structure(yaml_content)
if logic_errors:
formatted = "\n".join(f"- {err}" for err in logic_errors)
return formatted
graph = yaml_content.get("graph") or {}
try:
_ensure_supported(graph)
except Exception as e:
return str(e)
return ""

161
check/check_workflow.py Executable file
View File

@ -0,0 +1,161 @@
import argparse
from typing import Any, Dict, List, Optional, Tuple
import yaml
from check import check_yaml
from utils.io_utils import read_yaml
def _node_ids(graph: Dict[str, Any]) -> List[str]:
nodes = graph.get("nodes", []) or []
ids: List[str] = []
for n in nodes:
nid = n.get("id")
if isinstance(nid, str):
ids.append(nid)
return ids
def _edge_list(graph: Dict[str, Any]) -> List[Dict[str, Any]]:
edges = graph.get("edges", []) or []
return [e for e in edges if isinstance(e, dict) and "from" in e and "to" in e]
def _analyze_graph(graph: Dict[str, Any], base_path: str, errors: List[str]) -> None:
# Majority voting graphs are skipped for start/end structure checks
is_mv = graph.get("is_majority_voting", False)
if is_mv:
return
nodes = _node_ids(graph)
node_set = set(nodes)
# Validate provided start/end (if any) reference existing nodes
# start = graph.get("start")
end = graph.get("end")
# if start is not None and start not in node_set:
# errors.append(f"{base_path}.start references unknown node id '{start}'")
# Normalize to list
if end is not None:
if isinstance(end, str):
end_list = [end]
elif isinstance(end, list):
end_list = end
else:
errors.append(f"{base_path}.end must be a string or list of strings")
return
# Check each node ID in the end list
for end_node_id in end_list:
if not isinstance(end_node_id, str):
errors.append(
f"{base_path}.end contains non-string element: {end_node_id}"
)
elif end_node_id not in node_set:
errors.append(
f"{base_path}.end references unknown node id '{end_node_id}'"
)
# Compute in/out degrees within this graph scope
indeg = {nid: 0 for nid in nodes}
outdeg = {nid: 0 for nid in nodes}
for e in _edge_list(graph):
frm = e.get("from")
to = e.get("to")
if frm in outdeg:
outdeg[frm] += 1
if to in indeg:
indeg[to] += 1
# sources = [nid for nid in nodes if indeg.get(nid, 0) == 0]
sinks = [nid for nid in nodes if outdeg.get(nid, 0) == 0]
# # Rule:
# # - A non-cyclic (sub)graph should have exactly one natural source AND exactly one natural sink.
# # - Otherwise (e.g., multiple sources/sinks or cycles -> none), require explicit start or end.
# has_unique_source = len(sources) == 1
# has_unique_sink = len(sinks) == 1
# if not (has_unique_source and has_unique_sink):
# if start is None and end is None:
# errors.append(
# f"{base_path}: graph lacks a unique natural start and end; specify 'start' or 'end' explicitly"
# )
if not (len(sinks) == 1):
if end is None:
errors.append(
f"{base_path}: graph lacks a unique natural end; specify 'end' explicitly"
)
# Recurse into subgraphs
for i, n in enumerate(graph.get("nodes", []) or []):
if isinstance(n, dict) and n.get("type") == "subgraph":
sub = n.get("config") or {}
if not isinstance(sub, dict):
errors.append(f"{base_path}.nodes[{i}].config must be object for subgraph nodes")
continue
sg_type = sub.get("type")
if sg_type == "config":
config_block = sub.get("config")
if not isinstance(config_block, dict):
errors.append(
f"{base_path}.nodes[{i}].config.config must be object when type=config"
)
continue
_analyze_graph(config_block, f"{base_path}.nodes[{i}].config.config", errors)
elif sg_type == "file":
file_block = sub.get("config")
if not (isinstance(file_block, dict) and isinstance(file_block.get("path"), str)):
errors.append(
f"{base_path}.nodes[{i}].config.config.path must be string when type=file"
)
else:
errors.append(
f"{base_path}.nodes[{i}].config.type must be 'config' or 'file'"
)
def check_workflow_structure(data: Any) -> List[str]:
errors: List[str] = []
if not isinstance(data, dict) or "graph" not in data:
return ["<root>.graph is required"]
graph = data["graph"]
if not isinstance(graph, dict):
return ["<root>.graph must be object"]
_analyze_graph(graph, "graph", errors)
return errors
def main():
parser = argparse.ArgumentParser(
description="Check workflow structure: unique natural start/end or explicit start/end per (sub)graph")
parser.add_argument("path", nargs="?", default="design_0.4.0.yaml", help="Path to YAML file")
parser.add_argument("--no-schema", action="store_true", help="Skip schema validation (0.4.0)")
parser.add_argument("--fn-module", dest="fn_module", default=None,
help="Module name or .py path where edge functions are defined (for schema validation)")
args = parser.parse_args()
data = read_yaml(args.path)
if not args.no_schema:
schema_errors = check_yaml.validate_design(data, set_defaults=True, fn_module_ref=args.fn_module)
if schema_errors:
print("Invalid schema:")
for e in schema_errors:
print(f"- {e}")
raise SystemExit(1)
logic_errors = check_workflow_structure(data)
if logic_errors:
print("Workflow issues:")
for e in logic_errors:
print(f"- {e}")
raise SystemExit(2)
else:
print("Workflow OK.")
if __name__ == "__main__":
main()

46
check/check_yaml.py Executable file
View File

@ -0,0 +1,46 @@
"""Lightweight schema validation leveraging typed config loaders."""
import argparse
from pathlib import Path
from typing import Any, List, Optional
from entity.configs import ConfigError, DesignConfig
from utils.io_utils import read_yaml
def validate_design(data: Any, set_defaults: bool = True, fn_module_ref: Optional[str] = None) -> List[str]:
"""Validate raw YAML data using the typed config loader.
Note: This function validates schema structure only, without resolving
environment variable placeholders like ${VAR}. This allows workflows to
be saved even when environment variables are not yet configured - they
will be resolved at runtime.
"""
try:
if not isinstance(data, dict):
raise ConfigError("YAML root must be a mapping", path="root")
# Use DesignConfig.from_dict directly to skip placeholder resolution
# Users may configure environment variables at runtime
DesignConfig.from_dict(data)
return []
except ConfigError as exc:
return [str(exc)]
def main() -> None:
parser = argparse.ArgumentParser(description="Validate workflow YAML structure against the typed config loader")
parser.add_argument("path", help="Path to the workflow YAML file")
args = parser.parse_args()
data = read_yaml(args.path)
errors = validate_design(data)
if errors:
print("Design validation failed:")
for err in errors:
print(f"- {err}")
raise SystemExit(1)
print("Design validation successful.")
if __name__ == "__main__":
main()

83
docs/user_guide/en/attachments.md Executable file
View File

@ -0,0 +1,83 @@
# Attachment & Artifact API Guide
> Audience: operators integrating directly with backend REST/WS endpoints. The Web UI already handles most scenarios.
Attachments are files that can be uploaded, downloaded, or registered during a session. Artifacts are events emitted when attachments change so clients can listen in real time. This guide summarizes the REST endpoints, WebSocket mirrors, and storage policies.
## 1. Upload & List
### 1.1 Upload file
`POST /api/uploads/{session_id}`
- Headers: `Content-Type: multipart/form-data`
- Form field: `file`
- Response:
```json
{
"attachment_id": "att_bxabcd",
"name": "spec.md",
"mime": "text/markdown",
"size": 12345
}
```
- Files land in `WareHouse/<session>/code_workspace/attachments/` and are recorded in `attachments_manifest.json`.
### 1.2 List attachments
`GET /api/uploads/{session_id}`
- Returns metadata for all attachments in the session (ID, file name, mime, size, source).
### 1.3 Reference in execution requests
- `POST /api/workflow/execute` or WebSocket `human_input` payloads can include `attachments: ["att_xxx"]`. You still must supply `task_prompt`, even when you only want file uploads.
## 2. Artifact Events & Downloads
### 2.1 Real-time artifact events
`GET /api/sessions/{session_id}/artifact-events`
- Query params: `after`, `wait_seconds`, `include_mime`, `include_ext`, `max_size`, `limit`.
- Response includes `events[]`, `next_cursor`, `has_more`, `timed_out`.
- Event sample:
```json
{
"artifact_id": "art_123",
"attachment_id": "att_456",
"node_id": "python_runner",
"path": "code_workspace/result.json",
"size": 2048,
"mime": "application/json",
"hash": "sha256:...",
"timestamp": 1732699900
}
```
- WebSocket emits the same data via `artifact_created`, so dashboard clients can subscribe live.
### 2.2 Download a single artifact
`GET /api/sessions/{session_id}/artifacts/{artifact_id}`
- Query: `mode=meta|stream`, `download=true|false`.
- `meta` → metadata only; `stream` → file content. Add `download=true` to include `Content-Disposition`.
- Small files may be returned as `data_uri` when the server enables it.
### 2.3 Download an entire session
`GET /api/sessions/{session_id}/download`
- Packages `WareHouse/<session>/` into a zip for batch download.
## 3. File Lifecycle
1. Upload stage: files go under `code_workspace/attachments/`, and the manifest records `source`, `workspace_path`, `storage`, etc.
2. Python nodes/tools can call `AttachmentStore.register_file()` to turn workspace files into attachments; `WorkspaceArtifactHook` syncs events.
3. By default we retain all attachments for post-run downloads. Set `MAC_AUTO_CLEAN_ATTACHMENTS=1` to delete the `attachments/` directory after the session completes.
4. WareHouse zip downloads do **not** delete originals; schedule your own archival/cleanup jobs.
## 4. Size & Security
- **Size limits**: No hard cap in backend; enforce via reverse proxy (`client_max_body_size`, `max_request_body_size`) or customize `AttachmentService.save_upload_file`.
- **File types**: MIME detection controls `MessageBlockType` (image/audio/video/file); filter via `include_mime` as needed.
- **Virus/sensitive data**: Clients should pre-scan uploads; you can also trigger scanning services after save.
- **Permissions**: Attachment APIs require the session ID. In production guard with proxy-layer auth or internal JWT checks to prevent unauthorized downloads.
## 5. FAQ
| Issue | Mitigation |
| --- | --- |
| Upload 413 Payload Too Large | Raise proxy limits or FastAPI `client_max_size`; confirm disk quota. |
| Download link 404 | Check `session_id` spelling (allowed chars: letters/digits/`_-`) and confirm the session hasn’t been purged. |
| Missing artifact events | Ensure WebSocket is connected or use `artifact-events` REST polling with the `after` cursor. |
| Attachment not visible in Python node | Verify `code_workspace/attachments/` hasn’t been cleaned and `_context[python_workspace_root]` is correct. |
## 6. Client Patterns
- **Web UI**: Use artifact long-polling or WebSocket to refresh lists in real time; offer a “download all” button once nodes finish.
- **CLI/automation**: After runs complete, call `/download` for the zip; if you need just a subset, combine `artifact-events` with `include_ext` filters.
- **Test rigs**: Script the upload/download flow to validate proxy limits and CORS before shipping.

111
docs/user_guide/en/config_schema_contract.md Executable file
View File

@ -0,0 +1,111 @@
# Config Schema API Contract
This reference explains how `/api/config/schema` and `/api/config/schema/validate` expose DevAll's dynamic config metadata so IDEs, frontend form builders, and CLI tools can scope schemas with breadcrumbs.
## 1. Endpoints
| Method & Path | Purpose |
| --- | --- |
| `POST /api/config/schema` | Returns the schema for a config node described by breadcrumbs. |
| `POST /api/config/schema/validate` | Validates a YAML/JSON document and optionally echoes the scoped schema. |
### 1.1 Request body (shared fields)
```json
{
"breadcrumbs": [
{"node": "DesignConfig", "field": "graph"},
{"node": "GraphConfig", "field": "nodes"},
{"node": "NodeConfig", "value": "model"}
]
}
```
- `node` (required): class name (`DesignConfig`, `GraphConfig`, `NodeConfig`, etc.) that must match the class reached so far.
- `field` (optional): child field to traverse. When omitted, the breadcrumb only asserts you remain on `node`.
- `value` (optional): use when the child class depends on a discriminator (e.g., node `type`). Supply the value as it would appear in YAML.
- `index` (optional int): reserved for list traversal; current configs rely on `value`/`field` for navigation.
### 1.2 `/schema` response
```json
{
"schemaVersion": "0.1.0",
"node": "NodeConfig",
"fields": [
{
"name": "id",
"typeHint": "str",
"required": true,
"description": "Unique node identifier"
},
{
"name": "type",
"typeHint": "str",
"required": true,
"enum": ["model", "python", "agent"],
"enumOptions": [
{"value": "model", "label": "LLM Node", "description": "Runs provider-backed models"}
]
}
],
"constraints": [...],
"breadcrumbs": [...],
"cacheKey": "f90d..."
}
```
- `fields`: serialized `ConfigFieldSpec` entries; nested targets include `childRoutes`.
- `constraints`: emitted from `collect_schema()` (mutual exclusivity, required combos, etc.).
- `cacheKey`: SHA-1 hash of `{node, breadcrumbs}` so clients can memoize responses.
### 1.3 `/schema/validate` payloads
Add `document` alongside breadcrumbs:
```json
{
"breadcrumbs": [{"node": "DesignConfig"}],
"document": """
name: demo
version: 0.4.0
workflow:
nodes: []
edges: []
"""
}
```
Responses:
- Valid document: `{ "valid": true, "schema": { ... } }`
- Config error:
```json
{
"valid": false,
"error": "field 'nodes' must not be empty",
"path": ["workflow", "nodes"],
"schema": { ... }
}
```
- Malformed YAML: HTTP 400 with `{ "message": "invalid_yaml", "error": "..." }`.
## 2. Breadcrumb Tips
- Begin with `{ "node": "DesignConfig" }`.
- Each hop’s `node` must match the current config class or the API returns HTTP 422.
- Use `field` to step into nested configs (graph → nodes → config, etc.).
- Use `value` for discriminator-based children (node `type`, tooling `type`, etc.).
- Non-navigable targets raise `field '<name>' on <node> is not navigable`.
## 3. CLI Helper
```bash
python run.py --inspect-schema --schema-breadcrumbs '[{"node":"DesignConfig","field":"graph"}]'
```
The CLI prints the same JSON as `/schema`, which is useful while editing `FIELD_SPECS` or debugging registries before exporting templates.
## 4. Frontend Pattern
1. Fetch base schema with `[{node: 'DesignConfig', field: 'graph'}]` to render the workflow form.
2. When users open nested modals (node config, tooling config, etc.), append breadcrumbs describing the path and refetch.
3. Cache responses using `cacheKey` + breadcrumbs.
4. Before saving, call `/schema/validate` to surface `error` + `path` inline.
## 5. Error Reference
| HTTP Code | Situation | Detail payload |
| --- | --- | --- |
| 400 | YAML parse failure | `{ "message": "invalid_yaml", "error": "..." }` |
| 422 | Breadcrumb resolution failure | `{ "message": "breadcrumb node 'X'..." }` |
| 200 + `valid=false` | Backend `ConfigError` | `{ "error": "...", "path": ["workflow", ...] }` |
| 200 + `valid=true` | Document OK | Schema echoed back for the requested breadcrumbs. |
Pair this contract with `FIELD_SPECS` to build schema-aware experiences without hardcoding config structures.

276
docs/user_guide/en/dynamic_execution.md Executable file
View File

@ -0,0 +1,276 @@
# Dynamic Execution Mode Guide
Dynamic execution mode enables parallel processing behavior defined at the edge level, supporting Map (fan-out) and Tree (fan-out + reduce) modes. When messages pass through edges configured with `dynamic`, the target node dynamically expands into multiple parallel instances based on split results.
## 1. Overview
| Mode | Description | Output | Use Cases |
|------|-------------|--------|-----------|
| **Map** | Fan-out execution, splits messages into multiple units for parallel processing | `List[Message]` (flattened results) | Batch processing, parallel queries |
| **Tree** | Fan-out + reduce, parallel processing followed by recursive group merging | Single `Message` | Long text summarization, hierarchical aggregation |
## 2. Configuration Structure
Dynamic configuration is defined on **edges**, not nodes:
```yaml
edges:
- from: Source Node
to: Target Node
trigger: true
carry_data: true
dynamic: # Edge-level dynamic execution config
type: map # map or tree
split: # Message splitting strategy
type: message # message | regex | json_path
# pattern: "..." # Required for regex mode
# json_path: "..." # Required for json_path mode
config: # Mode-specific config
max_parallel: 5 # Maximum concurrency
```
### 2.1 Core Concepts
- **Dynamic edge**: An edge with `dynamic` configured; messages passing through trigger dynamic expansion of the target node
- **Static edge**: An edge without `dynamic` configured; messages are **replicated** to all dynamic expansion instances
- **Target node expansion**: The target node is "virtually" expanded into multiple parallel instances based on split results
### 2.2 Multi-Edge Consistency Rule
> [!IMPORTANT]
> When a node has multiple incoming edges with `dynamic` configured, all dynamic edge configurations **must be identical** (type, split, config). Otherwise, execution will fail with an error.
## 3. Split Strategies
Split defines how messages passing through the edge are partitioned into parallel execution units.
### 3.1 message Mode (Default)
Each message passing through the edge becomes an independent execution unit. This is the most common mode.
```yaml
split:
type: message
```
**Execution behavior**:
- Source node outputs 4 messages through the dynamic edge
- Splits into 4 parallel units, target node executes 4 times
### 3.2 regex Mode
Uses regular expressions to extract matches from text content.
```yaml
split:
type: regex
pattern: "(?s).{1,2000}(?:\\s|$)" # Split every ~2000 characters
```
**Typical uses**:
- Split by paragraph: `pattern: "\\n\\n"`
- Split by line: `pattern: ".+"`
- Fixed-length chunks: `pattern: "(?s).{1,N}"`
### 3.3 json_path Mode
Extracts array elements from JSON-formatted output using JSONPath expressions.
```yaml
split:
type: json_path
json_path: "$.items[*]" # JSONPath expression
```
## 4. Map Mode Details
Map mode splits messages and executes the target node in parallel, flattening outputs into `List[Message]`.
### 4.1 Configuration
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `max_parallel` | int | 10 | Maximum concurrent executions |
### 4.2 Execution Flow
```mermaid
flowchart LR
Source["Source Node Output"] --> Edge["Dynamic Edge (map)"]
Edge --> Split["Split"]
Split --> U1["Unit 1"]
Split --> U2["Unit 2"]
Split --> U3["Unit N"]
U1 --> P1["Target Node #1"]
U2 --> P2["Target Node #2"]
U3 --> P3["Target Node #N"]
P1 --> Merge["Merge Results"]
P2 --> Merge
P3 --> Merge
Merge --> Output["List[Message]"]
```
## 5. Tree Mode Details
Tree mode adds reduction layers on top of Map, recursively merging parallel results by group until a single output remains.
### 5.1 Configuration
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `group_size` | int | 3 | Number of elements per reduction group, minimum 2 |
| `max_parallel` | int | 10 | Maximum concurrent executions per layer |
### 5.2 Execution Flow
```mermaid
flowchart TB
subgraph Layer1["Layer 1: Parallel Execution"]
I1["Unit 1"] --> R1["Result 1"]
I2["Unit 2"] --> R2["Result 2"]
I3["Unit 3"] --> R3["Result 3"]
I4["Unit 4"] --> R4["Result 4"]
I5["Unit 5"] --> R5["Result 5"]
I6["Unit 6"] --> R6["Result 6"]
end
subgraph Layer2["Layer 2: Group Reduction (group_size=3)"]
R1 & R2 & R3 --> G1["Reduction Group 1"]
R4 & R5 & R6 --> G2["Reduction Group 2"]
end
subgraph Layer3["Layer 3: Final Reduction"]
G1 & G2 --> Final["Final Result"]
end
```
## 6. Static Edge Message Replication
When a target node has both dynamic and static incoming edges:
- **Dynamic edge messages**: Split according to strategy, each unit executes the target node once
- **Static edge messages**: **Replicated** to every dynamic expansion instance
```yaml
nodes:
- id: Task Generator
type: passthrough
config: ...
- id: Extra Requirement
type: literal
config:
content: "Please use concise language"
- id: Processor
type: agent
config:
name: gpt-4o
role: Process task
edges:
- from: Task Generator
to: Processor
dynamic: # Dynamic edge: 4 tasks → 4 parallel units
type: map
split:
type: message
config:
max_parallel: 10
- from: Extra Requirement
to: Processor # Static edge: replicated to all 4 instances
trigger: true
carry_data: true
```
**Execution result**: Processor executes 4 times, each receiving 1 task + "Please use concise language"
## 7. Complete Examples
### 7.1 Travel Planning (Map + Tree Combination)
```yaml
graph:
nodes:
- id: Eat Planner
type: literal
config:
content: Plan what to eat in Shanghai
role: user
- id: Play Planner
type: literal
config:
content: Plan what to do in Shanghai
role: user
- id: Stay Planner
type: literal
config:
content: Plan where to stay in Shanghai
role: user
- id: Collector
type: passthrough
config:
only_last_message: false
- id: Travel Executor
type: agent
config:
name: gpt-4o
role: You are a travel planner. Please plan according to user requests.
- id: Final Aggregator
type: agent
config:
name: gpt-4o
role: Please integrate the inputs into a complete travel plan.
edges:
- from: Eat Planner
to: Collector
- from: Play Planner
to: Collector
- from: Stay Planner
to: Collector
- from: Collector
to: Travel Executor
dynamic: # Map fan-out: 3 planning requests → 3 parallel executions
type: map
split:
type: message
config:
max_parallel: 10
- from: Travel Executor
to: Final Aggregator
dynamic: # Tree reduce: 3 results → 1 final plan
type: tree
split:
type: message
config:
group_size: 2
max_parallel: 10
```
### 7.2 Long Document Summarization (Tree Mode)
```yaml
edges:
- from: Document Source
to: Summarizer
dynamic:
type: tree
split:
type: regex
pattern: "(?s).{1,2000}(?:\\s|$)" # 2000-character chunks
config:
group_size: 3
max_parallel: 10
```
## 8. Performance Tips
- **Control concurrency**: Set reasonable `max_parallel` to avoid API rate limiting
- **Optimize split granularity**: Too fine increases overhead, too coarse limits parallelism
- **Tree group size**: `group_size=2-4` is usually optimal
- **Monitor costs**: Dynamic mode significantly increases API calls
## 9. Related Documentation
- [Edge Configuration Guide](../edges.md)
- [Workflow Authoring Guide](../workflow_authoring.md)
- [Agent Node Configuration](../nodes/agent.md)

229
docs/user_guide/en/execution_logic.md Executable file
View File

@ -0,0 +1,229 @@
# Graph Execution Logic
> Version: 2025-12-16
This document explains how the DevAll backend parses and executes workflow graphs, with particular focus on handling complex graphs containing cyclic structures.
## 1. Execution Engine Overview
The DevAll workflow execution engine supports two types of graph structures:
| Graph Type | Characteristics | Execution Strategy |
|------------|-----------------|-------------------|
| **DAG (Directed Acyclic Graph)** | No cyclic dependencies between nodes | Topological sort + parallel layer execution |
| **Cyclic Directed Graph** | Contains one or more loop structures | Recursive super-node scheduling |
The execution engine automatically detects the graph structure and selects the appropriate execution strategy.
## 2. DAG Execution Flow
For workflow graphs without cycles, the engine uses standard DAG scheduling:
1. **Build predecessor/successor relationships**: Parse edge definitions to establish `predecessors` and `successors` lists for each node
2. **Calculate in-degrees**: Count the number of predecessors for each node
3. **Topological sort**: Place nodes with in-degree 0 in the first layer; after execution, decrement successor in-degrees; new zero in-degree nodes enter the next layer
4. **Parallel layer execution**: Nodes within the same layer have no dependencies and can execute concurrently
```mermaid
flowchart LR
subgraph Layer1["Execution Layer 1"]
A["Node A"]
B["Node B"]
end
subgraph Layer2["Execution Layer 2"]
C["Node C"]
end
subgraph Layer3["Execution Layer 3"]
D["Node D"]
end
A --> C
B --> C
C --> D
```
## 3. Cyclic Graph Execution Flow
### 3.1 Tarjan's Strongly Connected Components Detection
When cyclic structures exist in the graph, the execution engine first uses **Tarjan's algorithm** to detect all Strongly Connected Components (SCCs). Tarjan's algorithm identifies all cycles in the graph in O(|V|+|E|) time complexity through depth-first search.
An SCC containing more than one node constitutes a cycle structure.
### 3.2 Super Node Construction
After detecting cycles, the execution engine abstracts each cycle into a "Super Node":
- All nodes within the cycle are encapsulated in the super node
- Dependencies between super nodes derive from cross-cycle edges between original nodes
- The resulting super node graph is guaranteed to be a DAG, enabling topological sorting
```mermaid
flowchart TB
subgraph Original["Original Graph"]
direction TB
A1["A"] --> B1["B"]
B1 --> C1["C"]
C1 --> B1
C1 --> D1["D"]
end
subgraph Abstracted["Super Node Graph"]
direction TB
A2["Node A"] --> S1["Super Node<br/>(B, C cycle)"]
S1 --> D2["Node D"]
end
Original -.->|"Abstract"| Abstracted
```
### 3.3 Recursive Cycle Execution Strategy
For cycle super nodes, the system employs a recursive execution strategy:
#### Step 1: Unique Initial Node Identification
Analyze the cycle boundary to identify the uniquely triggered entry node as the "initial node". This node must satisfy:
- Triggered by a predecessor node outside the cycle via a condition-satisfying edge
- Exactly one node meets this criterion
#### Step 2: Build Scoped Subgraph
Using all nodes in the current cycle as the scope, **logically remove all incoming edges to the initial node**. This operation breaks the outer cycle boundary, ensuring subsequent cycle detection only targets nested structures within.
#### Step 3: Nested Cycle Detection
Apply Tarjan's algorithm again to the subgraph to detect nested cycles within the scope. Since the initial node's incoming edges are removed, detected SCCs represent only true inner nested cycles.
#### Step 4: Inner Super Node Construction and Topological Sort
If nested cycles are detected:
- Abstract each inner cycle as a super node
- Build a super node dependency graph within the scope
- Perform topological sort on this super node graph
If no nested cycles are detected, perform direct DAG topological sort.
#### Step 5: Layered Execution
Execute according to the topological order:
- **Regular nodes**: Execute after checking trigger state; initial node executes unconditionally in the first iteration
- **Inner cycle super nodes**: **Recursively invoke Steps 1-6**, forming a nested execution structure
#### Step 6: Exit Condition Check
After completing each round of in-cycle execution, the system checks these exit conditions:
- **Exit edge triggered**: If any in-cycle node triggers an edge to an out-of-cycle node, exit the loop
- **Maximum iterations reached**: If the configured maximum (default 100) is reached, force termination
- **Initial node not re-triggered**: If the initial node isn't re-triggered by in-cycle predecessors, the loop naturally terminates
If none of the conditions are met, return to Step 2 for the next iteration.
### 3.4 Cycle Execution Flowchart
```mermaid
flowchart TB
A["Cycle super node scheduled"] --> B["Identify uniquely triggered initial node"]
B --> C{"Valid initial node?"}
C -->|"None"| D["Skip this cycle"]
C -->|"Multiple"| E["Report configuration error"]
C -->|"Unique"| F["Build scoped subgraph<br/>Remove initial node's incoming edges"]
F --> G["Tarjan algorithm: detect nested cycles"]
G --> H{"Inner nested cycles exist?"}
H -->|"No"| I["DAG topological sort"]
H -->|"Yes"| J["Build inner super nodes<br/>Topological sort"]
I --> K["Layered execution"]
J --> K
K --> L["Execute regular nodes"]
K --> M["Recursively execute inner cycles"]
L --> N{"Check exit conditions"}
M --> N
N -->|"Exit edge triggered"| O["Exit cycle"]
N -->|"Max iterations reached"| O
N -->|"Initial node not re-triggered"| O
N -->|"Continue iteration"| F
```
## 4. Edge Conditions and Trigger Mechanism
### 4.1 Edge Trigger
Each edge has a `trigger` attribute that determines whether it participates in execution order calculation:
| trigger value | Behavior |
|---------------|----------|
| `true` (default) | Edge participates in topological sort; target node waits for source completion |
| `false` | Edge doesn't participate in topological sort; used only for data transfer |
### 4.2 Edge Condition
Edge conditions determine whether data flows along the edge:
- `true` (default): Always transfer
- `keyword`: Check if upstream output contains/excludes specific keywords
- `function`: Invoke custom function for evaluation
- Other custom condition types
The target node is triggered for execution only when the condition is satisfied.
## 5. Typical Cycle Scenario Examples
### 5.1 Human Review Loop
```yaml
nodes:
- id: Writer
type: agent
config:
name: gpt-4o
role: You are a professional technical writer
- id: Reviewer
type: human
config:
description: Please review the article, enter ACCEPT if satisfied
edges:
- from: Writer
to: Reviewer
- from: Reviewer
to: Writer
condition:
type: keyword
config:
none: [ACCEPT] # Continue loop when ACCEPT is not present
```
Execution flow:
1. Writer generates article
2. Reviewer performs human review
3. If input doesn't contain "ACCEPT", return to Writer for revision
4. If input contains "ACCEPT", exit the loop
### 5.2 Nested Loops
The system supports arbitrarily deep nested loops. For example, an outer "review-revise" loop can contain an inner "generate-validate" loop:
```
Outer Loop (Writer -> Reviewer -> Writer)
└── Inner Loop (Generator -> Validator -> Generator)
```
The recursive execution strategy automatically handles such nested structures.
## 6. Key Code Modules
| Module | Function |
|--------|----------|
| `workflow/cycle_manager.py` | Tarjan algorithm implementation, cycle info management |
| `workflow/topology_builder.py` | Super node graph construction, topological sorting |
| `workflow/executor/cycle_executor.py` | Recursive cycle executor |
| `workflow/graph.py` | Main graph execution entry point |
## 7. Changelog
- **2025-12-16**: Added graph execution logic documentation, detailing DAG and cyclic graph execution strategies.

61
docs/user_guide/en/field_specs.md Executable file
View File

@ -0,0 +1,61 @@
# FIELD_SPECS Authoring Guide
This guide explains how to write `FIELD_SPECS` for new configs so the Web UI forms and `python -m tools.export_design_template` stay in sync. It applies to any `BaseConfig` subclass (nodes, Memory, Thinking, Tooling, etc.).
## 1. Why FIELD_SPECS matter
- UI forms rely on `FIELD_SPECS` to render inputs, defaults, and helper text.
- The design template exporter reads `FIELD_SPECS` to populate `yaml_template/design*.yaml` and the mirrored files under `frontend/public/`.
- Fields missing from `FIELD_SPECS` will not appear in the UI or generated templates.
## 2. Basic structure
`FIELD_SPECS` is a dict mapping field name → `ConfigFieldSpec`, usually declared inside the config class:
```python
FIELD_SPECS = {
"interpreter": ConfigFieldSpec(
name="interpreter",
display_name="Interpreter",
type_hint="str",
required=False,
default="python3",
description="Path to the Python executable",
),
...
}
```
Key attributes:
- `name`: same as the YAML field.
- `display_name`: optional human label shown in the UI; falls back to `name` when omitted.
- `type_hint`: string describing the shape (`str`, `list[str]`, `dict[str, Any]`, etc.).
- `required`: whether the UI treats the field as mandatory; usually `False` if a default exists.
- `default`: scalar or JSON-serializable default value.
- `description`: helper text/tooltips in forms and docs.
- `enum`: array of allowed values (strings).
- `enumOptions`: richer metadata per enum entry (label, description). Use it alongside `enum` for user-friendly dropdowns.
- `child`: reference to another `BaseConfig` subclass for nested structures.
## 3. Authoring flow
1. **Validate in `from_dict`** – ensure YAML parsing enforces types and emits clear `ConfigError`s (see `entity/configs/python_runner.py`).
2. **Define `FIELD_SPECS`** – cover all public fields with type, description, default, etc.
3. **Handle dynamic fields** – when options depend on registries or filesystem scans, override `field_specs()` and use `replace()` to inject real-time `enum`/`description` (e.g., `FunctionToolEntryConfig.field_specs()` lists functions on disk).
4. **Export templates** – after edits run:
```bash
python -m tools.export_design_template --output yaml_template/design.yaml --mirror frontend/public/design.yaml
```
The command uses the latest `FIELD_SPECS` to regenerate YAML templates and the frontend mirror—no manual edits needed.
## 4. Common patterns
- **Scalar fields**: see `entity/configs/python_runner.py` for `timeout_seconds` (integer default + validation).
- **Nested lists**: `entity/configs/memory.py` uses `child=FileSourceConfig` for `file_sources`, enabling repeatable subforms.
- **Dynamic enums**: `Node.field_specs()` (around `entity/configs/node.py:304`) pulls node types from the registry and supplies `enumOptions`; `FunctionToolEntryConfig.field_specs()` builds enumerations from the function catalog.
- **Registry-driven descriptions**: when calling `register_node_type` / `register_memory_store` / `register_thinking_mode` / `register_tooling_type`, always provide `summary`/`description`. Those strings populate `enumOptions` and keep dropdowns self-explanatory.
- **Optional blocks**: combine `required=False` with sensible `default` values and make sure `from_dict` honors the same defaults.
## 5. Best practices
- Keep descriptions user-friendly and clarify units (e.g., “Timeout (seconds)”).
- Align defaults with parsing logic so UI expectations match backend behavior.
- For nested configs, provide concise examples or cross-links so UI users understand the structure.
- After changing `FIELD_SPECS`, re-run the export command and commit the updated templates/mirror files.
For more examples inspect `entity/configs/model.py`, `entity/configs/tooling.py`, or other existing node/Memory/Thinking configs.

45
docs/user_guide/en/index.md Executable file
View File

@ -0,0 +1,45 @@
# DevAll Backend User Guide (English)
This landing page helps operators, workflow authors, and extension developers find the right documentation for DevAll backend components. Use the table below as your navigation map; drill into the linked chapters for full procedures and examples.
## 1. Documentation Map
| Topic | Highlights |
| --- | --- |
| [Web UI Quick Start](web_ui_guide.md) | Frontend interface operations, workflow execution, human review, troubleshooting. |
| [Workflow Authoring](workflow_authoring.md) | YAML structure, node types, provider/edge conditions, template export, CLI execution. |
| [Graph Execution Logic](execution_logic.md) | DAG/cyclic graph execution, Tarjan cycle detection, super node construction, recursive cycle execution. |
| [Dynamic Parallel Execution](dynamic_execution.md) | Map/Tree modes, split strategies, parallel processing and hierarchical reduction. |
| [Memory Module](modules/memory.md) | Memory list architecture, built-in `simple`/`file`/`blackboard` behaviors, embedding config, troubleshooting. |
| [Thinking Module](modules/thinking.md) | Reasoning enhancement, self-reflection mode, extending custom thinking modes. |
| [Tooling Module](modules/tooling/README.md) | Function/MCP modes, context injection, built-in function catalog, MCP launch patterns. |
| [Node Types Reference](nodes/) | Agent, Python, Human, Subgraph, Passthrough, Literal, Loop Counter node configurations. |
| [Attachment & Artifact APIs](attachments.md) | Upload/list/download endpoints, manifest schema, cleanup strategies, security constraints. |
| [`FIELD_SPECS` Standard](field_specs.md) | Field metadata contract that powers UI forms and template export—required reading before customizing modules. |
| [Config Schema API Contract](config_schema_contract.md) | `/api/config/schema(*)` request examples and breadcrumbs protocol (mostly for frontend/IDE integrations). |
## 2. Product Overview (Backend Focus)
- **Workflow orchestration engine**: Parses YAML DAGs, coordinates `model`, `python`, `tooling`, and `human` nodes inside a shared context, and writes node outputs into `WareHouse/<session>/`.
- **Provider abstraction**: The `runtime/node/agent/providers/` layer encapsulates OpenAI, Gemini, and other APIs so each node can swap models, base URLs, credentials, plus optional `thinking` and `memories` settings.
- **Real-time observability**: FastAPI + WebSocket streams node states, stdout/stderr, and artifact events to the Web UI, while structured logs land in `logs/` for centralized collection.
- **Run asset management**: Every execution creates an isolated session; attachments, Python workspace, context snapshots, and summary outputs are downloadable for later review.
## 3. Architecture & Execution Flow
1. **Entry**: Web UI and CLI call the FastAPI server exposed via `server_main.py` (e.g., `/api/workflow/execute`).
2. **Validation/queueing**: `WorkflowRunService` validates YAML, creates a session, prepares `code_workspace/attachments/`, then hands the DAG to the scheduler in `workflow/`.
3. **Execution**: Node executors resolve dependencies, propagate context, call tools, and retrieve memories; `MemoryManager`, `ToolingConfig`, and `ThinkingManager` trigger inside agent nodes as needed.
4. **Observability**: WebSocket pushes states, logs, and artifact events; JSON logs stay in `logs/`, and `WareHouse/` stores run assets.
5. **Cleanup & download**: After completion you can bundle the session for download or fetch files individually via the attachment APIs; retention policies are deployment-specific.
## 4. Role-based Navigation
- **Solutions/Prompt Engineers**: Begin with [Workflow Authoring](workflow_authoring.md); read the Memory and Tooling module docs when you need context memories or custom tools.
- **Extension Developers**: Combine [`FIELD_SPECS`](field_specs.md) with the [Tooling module guide](modules/tooling/README.md) to register new components; reference [config_schema_contract.md](config_schema_contract.md) if you need to debug schema-driven UI.
## 5. Glossary
- **Session**: Unique ID (timestamp + name) for a single run, used across the Web UI, backend, and `WareHouse/`.
- **code_workspace**: Shared directory for Python nodes at `WareHouse/<session>/code_workspace/`, synchronized with relevant attachments.
- **Attachment**: Files uploaded by users or generated during runs; list/download via REST or WebSocket APIs.
- **Memory Store / Attachment**: Stores define persistence backends; memory attachments describe how agent nodes read/write those stores across phases.
- **Tooling**: Execution environment bound to agent nodes (Function or MCP implementations).
If you spot gaps or outdated instructions, open an issue/PR or edit the docs directly (remember to keep Chinese and English versions in sync).

117
docs/user_guide/en/modules/memory.md Executable file
View File

@ -0,0 +1,117 @@
# Memory Module Guide
This document explains DevAll's memory system: memory list config, built-in store implementations, how agent nodes attach memories, and troubleshooting tips. Core code lives in `entity/configs/memory.py` and `node/agent/memory/*.py`.
## 1. Architecture
1. **Memory Store** – Declared under `memory[]` in YAML with `name`, `type`, and `config`. Types are registered via `register_memory_store()` and point to concrete implementations.
2. **Memory Attachment** – Referenced inside agent nodes via `AgentConfig.memories`. Each `MemoryAttachmentConfig` defines read/write strategy and retrieval stages.
3. **MemoryManager** – Builds store instances at runtime based on attachments and orchestrates `load()`, `retrieve()`, `update()`, `save()`.
4. **Embedding** – `SimpleMemoryConfig` and `FileMemoryConfig` embed `EmbeddingConfig`, and `EmbeddingFactory` instantiates OpenAI or local vector models.
## 2. Memory Sample Config
```yaml
memory:
- name: convo_cache
type: simple
config:
memory_path: WareHouse/shared/simple.json
embedding:
provider: openai
model: text-embedding-3-small
api_key: ${API_KEY}
- name: project_docs
type: file
config:
index_path: WareHouse/index/project_docs.json
file_sources:
- path: docs/
file_types: [".md", ".mdx"]
recursive: true
embedding:
provider: openai
model: text-embedding-3-small
```
## 3. Built-in Store Comparison
| Type | Path | Highlights | Best for |
| --- | --- | --- | --- |
| `simple` | `node/agent/memory/simple_memory.py` | Optional disk persistence (JSON) after runs; FAISS + semantic rerank; read/write capable. | Small conversation history, prototypes. |
| `file` | `node/agent/memory/file_memory.py` | Chunks files/dirs into a vector index, read-only, auto rebuilds when files change. | Knowledge bases, doc QA. |
| `blackboard` | `node/agent/memory/blackboard_memory.py` | Lightweight append-only log trimmed by time/count; no vector search. | Broadcast boards, pipeline debugging. |
All stores register through `register_memory_store()` so summaries show up in UI via `MemoryStoreConfig.field_specs()`.
## 4. MemoryAttachmentConfig Fields
| Field | Description |
| --- | --- |
| `name` | Target Memory Store name (must be unique inside `stores[]`). |
| `retrieve_stage` | Optional list limiting retrieval to certain `AgentExecFlowStage` values (`pre`, `plan`, `gen`, `critique`, etc.). Empty means all stages. |
| `top_k` | Number of items per retrieval (default 3). |
| `similarity_threshold` | Minimum similarity cutoff (`-1` disables filtering). |
| `read` / `write` | Whether this node can read from / write back to the store. |
Agent node example:
```yaml
nodes:
- id: answer
type: agent
config:
provider: openai
model: gpt-4o-mini
prompt_template: answer_user
memories:
- name: convo_cache
retrieve_stage: ["gen"]
top_k: 5
read: true
write: true
- name: project_docs
read: true
write: false
```
Execution order:
1. When the node enters `gen`, `MemoryManager` iterates attachments.
2. Attachments matching the stage and `read=true` call `retrieve()` on their store.
3. Retrieved items are formatted under a "===== Related Memories =====" block in the agent context.
4. After completion, attachments with `write=true` call `update()` and optionally `save()`.
## 5. Store Details
All memory stores persist a unified `MemoryItem` structure containing:
- `content_summary` – trimmed text used for embedding search.
- `input_snapshot` / `output_snapshot` – serialized message blocks (with base64 attachments) preserving multimodal context.
- `metadata` – store-specific telemetry (role, previews, attachment IDs, etc.).
This schema lets multimodal outputs flow into Memory/Thinking modules without extra plumbing.
### 5.1 SimpleMemory
- **Path** – `SimpleMemoryConfig.memory_path` (or `auto`). Defaults to in-memory.
- **Retrieval** – Build a query from the prompt, trim it, embed, query FAISS `IndexFlatIP`, then apply semantic rerank (Jaccard/LCS).
- **Write** – `update()` builds a `MemoryContentSnapshot` (text + blocks) for both input/output, deduplicates via hashed summary, embeds the summary, and stores the snapshots/attachments metadata.
- **Tips** – Tune `max_content_length`, `top_k`, and `similarity_threshold` to avoid irrelevant context.
### 5.2 FileMemory
- **Config** – Requires at least one `file_sources` entry (paths, suffix filters, recursion, encoding). `index_path` is mandatory for incremental updates.
- **Indexing** – Scan files → chunk (default 500 chars, 50 overlap) → embed → persist JSON with `file_metadata`.
- **Retrieval** – Uses FAISS cosine similarity. Read-only; `update()` unsupported.
- **Maintenance** – `load()` checks file hashes and rebuilds if needed. Store `index_path` on persistent storage.
### 5.3 BlackboardMemory
- **Config** – `memory_path` (or `auto`) plus `max_items`. Creates the file in the session directory if missing.
- **Retrieval** – Returns the latest `top_k` entries ordered by time.
- **Write** – `update()` appends the latest snapshot (input/output blocks, attachments, previews). No embeddings are generated, so retrieval is purely recency-based.
## 6. EmbeddingConfig Notes
- Fields: `provider`, `model`, `api_key`, `base_url`, `params`.
- `provider=openai` uses the official client; override `base_url` for compatibility layers.
- `params` can include `use_chunking`, `chunk_strategy`, `max_length`, etc.
- `provider=local` expects `params.model_path` and depends on `sentence-transformers`.
## 7. Troubleshooting & Best Practices
- **Duplicate names** – The memory list enforces unique `memory[]` names. Duplicates raise `ConfigError`.
- **Missing embeddings** – `SimpleMemory` without embeddings downgrades to append-only; `FileMemory` errors out. Provide an embedding config whenever semantic search is required.
- **Permissions** – Ensure directories for `memory_path`/`index_path` are writable. Mount volumes when running inside containers.
- **Performance** – Pre-build large `FileMemory` indexes offline, use `retrieve_stage` to limit retrieval frequency, and tune `top_k`/`similarity_threshold` to balance recall vs. token cost.
## 8. Extending Memory
1. Implement a Config + Store (subclass `MemoryBase`).
2. Register via `register_memory_store("my_store", config_cls=..., factory=..., summary="...")` in `node/agent/memory/registry.py`.
3. Add `FIELD_SPECS`, then run `python -m tools.export_design_template ...` so the frontend picks up the enum.
4. Update this guide or ship a README detailing configuration knobs and boundaries.

108
docs/user_guide/en/modules/thinking.md Executable file
View File

@ -0,0 +1,108 @@
# Thinking Module Guide
The Thinking module provides reasoning enhancement capabilities for Agent nodes, enabling the model to perform additional inference before or after generating results. This document covers the Thinking module architecture, built-in modes, and configuration methods.
## 1. Architecture
1. **ThinkingConfig**: Declared in YAML under `nodes[].config.thinking`, containing `type` and `config` fields.
2. **ThinkingManagerBase**: Abstract base class defining thinking logic for two timing hooks: `_before_gen_think` and `_after_gen_think`.
3. **Registry**: New thinking modes are registered via `register_thinking_mode()`, and Schema API automatically displays available options.
## 2. Configuration Example
```yaml
nodes:
- id: Thoughtful Agent
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
thinking:
type: reflection
config:
reflection_prompt: |
Please carefully review your response, considering:
1. Is the logic sound?
2. Are there any factual errors?
3. Is the expression clear?
Then provide an improved response.
```
## 3. Built-in Thinking Modes
| Type | Description | Trigger Timing | Config Fields |
|------|-------------|----------------|---------------|
| `reflection` | Model reflects on and refines its output after generation | After generation (`after_gen`) | `reflection_prompt` |
### 3.1 Reflection Mode
Self-Reflection mode allows the model to reflect on and improve its initial output. The execution flow:
1. Agent node calls the model to generate initial response
2. ThinkingManager concatenates conversation history (system role, user input, model output) as reflection context
3. Calls the model again with `reflection_prompt` to generate reflection result
4. Reflection result replaces the original output as the final node output
#### Configuration
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `reflection_prompt` | string | Yes | Prompt guiding model reflection, specifying reflection dimensions and expected improvements |
#### Use Cases
- **Writing refinement**: Self-review and correct grammar, logic issues
- **Code review**: Automatic security and quality checks after code generation
- **Complex reasoning**: Verify and correct multi-step reasoning results
## 4. Execution Timing
ThinkingManager supports two execution timings:
| Timing | Property | Description |
|--------|----------|-------------|
| Before generation (`before_gen`) | `before_gen_think_enabled` | Execute thinking before model call for input preprocessing |
| After generation (`after_gen`) | `after_gen_think_enabled` | Execute thinking after model output for post-processing or refinement |
The built-in `reflection` mode only enables after-generation thinking. Extension developers can implement before-generation thinking as needed.
## 5. Interaction with Memory
The Thinking module can access Memory context:
- `ThinkingPayload.text`: Text content at current stage
- `ThinkingPayload.blocks`: Multimodal content blocks (images, attachments, etc.)
- `ThinkingPayload.metadata`: Additional metadata
Memory retrieval results are passed to thinking functions via the `memory` parameter, allowing reflection to reference historical memories.
## 6. Custom Thinking Mode Extension
1. **Create config class**: Inherit from `BaseConfig`, define required configuration fields
2. **Implement ThinkingManager**: Inherit from `ThinkingManagerBase`, implement `_before_gen_think` or `_after_gen_think`
3. **Register mode**:
```python
from runtime.node.agent.thinking.registry import register_thinking_mode
register_thinking_mode(
"my_thinking",
config_cls=MyThinkingConfig,
manager_cls=MyThinkingManager,
summary="Custom thinking mode description",
)
```
4. **Export template**: Run `python -m tools.export_design_template` to update frontend options
## 7. Best Practices
- **Control reflection rounds**: Current reflection is single-round; specify iteration requirements in `reflection_prompt` for multi-round
- **Concise prompts**: Lengthy `reflection_prompt` increases token consumption; focus on key improvement points
- **Combine with Memory**: Store important reflection results in Memory for downstream nodes
- **Monitor costs**: Reflection makes additional model calls; track token usage
## 8. Related Documentation
- [Agent Node Configuration](../nodes/agent.md)
- [Memory Module](memory.md)
- [Workflow Authoring Guide](../workflow_authoring.md)

47
docs/user_guide/en/modules/tooling/README.md Executable file
View File

@ -0,0 +1,47 @@
# Tooling Module Overview
DevAll currently exposes two tool binding modes for agent nodes:
1. **Function Tooling** – call in-repo Python functions from `functions/function_calling/`, with JSON Schema auto-generated from type hints.
2. **MCP Tooling** – connect to external services that implement the Model Context Protocol, including FastMCP, Claude Desktop, or any MCP-compatible tool stack.
All tooling configs hang off `AgentConfig.tooling`:
```yaml
nodes:
- id: solve
type: agent
config:
provider: openai
model: gpt-4o-mini
prompt_template: solver
tooling:
type: function
config:
tools:
- name: describe_available_files
- name: load_file
auto_load: true
timeout: 20
```
## 1. Lifecycle
1. **Parse** – `ToolingConfig` selects `FunctionToolConfig`, `McpRemoteConfig`, or `McpLocalConfig` based on `type`. Field definitions live in `entity/configs/tooling.py`.
2. **Runtime** – When the LLM chooses a tool, the executor injects `_context` (attachment store, workspace paths, etc.) for Function tools or forwards the request through MCP.
3. **Completion** – Tool outputs are appended to the agent message stream and, when relevant, registered as attachments (e.g., `load_file`).
## 2. Documentation Map
- [function.md](function.md) – Function Tooling config, context injection, best practices.
- [function_catalog.md](function_catalog.md) – Built-in function list with usage notes.
- [mcp.md](mcp.md) – MCP Tooling config, auto-launch, FastMCP example, security guidance.
## 3. Quick Comparison
| Dimension | Function | MCP |
| --- | --- | --- |
| Deployment | In-process Python functions shipped with the backend. | Remote: call an HTTP MCP endpoint. Local: launch a process and talk over stdio. |
| Schemas | Derived from annotations + `ParamMeta`. | Provided by the MCP server's JSON Schema. |
| Context | `_context` provides attachments + workspace helpers automatically. | Depends on the MCP server implementation. |
| Typical use | File I/O, local scripts, internal APIs. | Third-party tool suites, browsers, database agents. |
## 4. Security Notes
- Function Tooling runs inside the backend process, so keep functions least-privileged and avoid executing arbitrary shell commands without validation.
- MCP Tooling now has explicit **remote (HTTP)** and **local (stdio)** modes. Remote only needs an existing server URL; Local launches your binary, so constrain the command/env vars and rely on `wait_for_log` + timeouts to detect readiness.
- Tools that mutate attachments or `code_workspace/` should respect the lifecycle described in the [Attachment guide](../../attachments.md) (Chinese for now) to avoid leaking artifacts.

77
docs/user_guide/en/modules/tooling/function.md Executable file
View File

@ -0,0 +1,77 @@
# Function Tooling Configuration Guide
`FunctionToolConfig` lets agent nodes call Python functions defined in the repo. Implementation lives in `entity/configs/tooling.py`, `utils/function_catalog.py`, and `functions/function_calling/`.
## 1. Config Fields
| Field | Description |
| --- | --- |
| `tools` | List of `FunctionToolEntryConfig`. Each entry requires `name`. |
| `timeout` | Tool execution timeout (seconds). |
`FunctionToolEntryConfig` specifics:
- `name`: top-level function name in `functions/function_calling/`.
### Function picker (`module_name:function_name`) & `module_name:All`
- The dropdown displays each function as `module_name:function_name`, where `module_name` is the relative Python file under `functions/function_calling/` (without `.py`, nested folders joined by `/`). This preserves semantic grouping for large catalogs.
- Every module automatically prepends a `module_name:All` entry, and all `All` entries are sorted lexicographically ahead of concrete functions. Choosing it expands to all functions in that module during config parsing, preserving alphabetical order.
- `module_name:All` is strictly for bulk imports; overriding `description`/`parameters`/`auto_fill` alongside it raises a validation error. Customize individual functions after expansion if needed.
- Both modules and functions are sorted alphabetically, and YAML still stores the plain function names; `module_name:All` is merely an input shortcut.
## 2. Function Directory Requirements
- Path: `functions/function_calling/` (override with `MAC_FUNCTIONS_DIR`).
- Functions must live at module top level.
- Provide Python type hints; for enums/descriptions use `typing.Annotated[..., ParamMeta(...)]`.
- Parameters beginning with `_` or splats (`*args`/`**kwargs`) are hidden from the agent call.
- The docstring’s first paragraph becomes the description (truncated to ~600 chars).
- `utils/function_catalog.py` builds JSON Schemas at startup for the frontend/CLI.
## 3. Context Injection
The executor passes `_context` into each function:
| Key | Value |
| --- | --- |
| `attachment_store` | `utils.attachments.AttachmentStore` for querying/registering attachments. |
| `python_workspace_root` | Session `code_workspace/` shared by Python nodes. |
| `graph_directory` | Session root directory for relative path helpers. |
| others | Environment-specific extras (session/node IDs, etc.). |
Functions can declare `_context: dict | None = None` and parse it (see `functions/function_calling/file.py`’s `FileToolContext`).
## 4. Example: Read Text File
```python
from typing import Annotated
from utils.function_catalog import ParamMeta
def read_text_file(
path: Annotated[str, ParamMeta(description="workspace-relative path")],
*,
encoding: str = "utf-8",
_context: dict | None = None,
) -> str:
ctx = FileToolContext(_context)
target = ctx.resolve_under_workspace(path)
return target.read_text(encoding=encoding)
```
YAML usage:
```yaml
nodes:
- id: summarize
type: agent
config:
tooling:
type: function
config:
tools:
- name: describe_available_files
- name: read_text_file
```
## 5. Extension Flow
1. Add your function under `functions/function_calling/`.
2. Supply type hints + `ParamMeta`; set `auto_fill: false` with custom `parameters` if you need manual JSON Schema.
3. If the function needs extra packages, declare them in `pyproject.toml`/`requirements.txt`, or use the bundled `install_python_packages` sparingly.
4. Run `python -m tools.export_design_template ...` so the frontend picks up new enums.
## 6. Debugging
- If the frontend/CLI reports function `foo` not found, double-check the name and ensure it resides under `MAC_FUNCTIONS_DIR`.
- When `function_catalog` fails to load, `FunctionToolEntryConfig.field_specs()` includes the error—fix syntax or dependencies first.
- Tool timeouts bubble up to the agent; raise `timeout` or handle exceptions inside the function for friendlier responses.

168
docs/user_guide/en/modules/tooling/function_catalog.md Executable file
View File

@ -0,0 +1,168 @@
# Built-in Function Tool Catalog
This document lists all preset tools in the `functions/function_calling/` directory for Agent nodes to use via Function Tooling.
## Quick Import
Reference tools in YAML as follows:
```yaml
tooling:
- type: function
config:
tools:
- name: file:All # Import entire module
- name: save_file # Import single function
- name: deep_research:All
```
---
## File Operations (file.py)
Tools for file and directory management within `code_workspace/`.
| Function | Description |
|----------|-------------|
| `describe_available_files` | List available files in attachment store and code_workspace |
| `list_directory` | List contents of a directory |
| `create_folder` | Create a folder (supports nested directories) |
| `delete_path` | Delete a file or directory |
| `load_file` | Load a file and register as attachment, supports multimodal (text/image/audio) |
| `save_file` | Save text content to a file |
| `read_text_file_snippet` | Read text snippet (offset + limit), suitable for large files |
| `read_file_segment` | Read file by line range, supports line number metadata |
| `apply_text_edits` | Apply multiple text edits while preserving newlines and encoding |
| `rename_path` | Rename a file or directory |
| `copy_path` | Copy a file or directory tree |
| `move_path` | Move a file or directory |
| `search_in_files` | Search for text or regex patterns in workspace files |
**Example YAML**: [ChatDev_v1.yaml](../../../../../yaml_instance/ChatDev_v1.yaml), [file_tool_use_case.yaml](../../../../../yaml_instance/file_tool_use_case.yaml)
---
## Python Environment Management (uv_related.py)
Manage Python environments and dependencies using uv.
| Function | Description |
|----------|-------------|
| `install_python_packages` | Install Python packages using `uv add` |
| `init_python_env` | Initialize Python environment (uv lock + venv) |
| `uv_run` | Execute uv run in workspace to run modules or scripts |
**Example YAML**: [ChatDev_v1.yaml](../../../../../yaml_instance/ChatDev_v1.yaml)
---
## Deep Research (deep_research.py)
Search result management and report generation tools for automated research workflows.
### Search Result Management
| Function | Description |
|----------|-------------|
| `search_save_result` | Save or update a search result (URL, title, abstract, details) |
| `search_load_all` | Load all saved search results |
| `search_load_by_url` | Load a specific search result by URL |
| `search_high_light_key` | Save highlighted keywords for a search result |
### Report Management
| Function | Description |
|----------|-------------|
| `report_read` | Read full report content |
| `report_read_chapter` | Read a specific chapter (supports multi-level paths like `Intro/Background`) |
| `report_outline` | Get report outline (header hierarchy) |
| `report_create_chapter` | Create a new chapter |
| `report_rewrite_chapter` | Rewrite chapter content |
| `report_continue_chapter` | Append content to an existing chapter |
| `report_reorder_chapters` | Reorder chapters |
| `report_del_chapter` | Delete a chapter |
| `report_export_pdf` | Export report to PDF |
**Example YAML**: [deep_research_v1.yaml](../../../../../yaml_instance/deep_research_v1.yaml)
---
## Web Tools (web.py)
Web search and webpage content retrieval.
| Function | Description |
|----------|-------------|
| `web_search` | Perform web search using Serper.dev, supports pagination and multiple languages |
| `read_webpage_content` | Read webpage content using Jina Reader, supports rate limiting |
**Environment Variables**:
- `SERPER_DEV_API_KEY`: Serper.dev API key
- `JINA_API_KEY`: Jina API key (optional, auto rate-limited to 20 RPM without key)
**Example YAML**: [deep_research_v1.yaml](../../../../../yaml_instance/deep_research_v1.yaml)
---
## Video Tools (video.py)
Manim animation rendering and video processing.
| Function | Description |
|----------|-------------|
| `render_manim` | Render Manim script, auto-detects scene class and outputs video |
| `concat_videos` | Concatenate multiple video files using FFmpeg |
**Example YAML**: [teach_video.yaml](../../../../../yaml_instance/teach_video.yaml), [teach_video.yaml](../../../../../yaml_instance/teach_video.yaml)
---
## Code Execution (code_executor.py)
| Function | Description |
|----------|-------------|
| `execute_code` | Execute Python code string, returns stdout and stderr |
> ⚠️ **Security Note**: This tool has elevated privileges and should only be used in trusted workflows.
---
## User Interaction (user.py)
| Function | Description |
|----------|-------------|
| `call_user` | Send instructions to the user and get a response, for scenarios requiring human input |
---
## Weather Query (weather.py)
Demo tools to illustrate Function Calling workflow.
| Function | Description |
|----------|-------------|
| `get_city_num` | Return city code (hardcoded example) |
| `get_weather` | Return weather info by city code (hardcoded example) |
---
## Adding Custom Tools
1. Create a Python file in `functions/function_calling/` directory
2. Define parameters using type annotations:
```python
from typing import Annotated
from utils.function_catalog import ParamMeta
def my_tool(
param1: Annotated[str, ParamMeta(description="Parameter description")],
*,
_context: dict | None = None, # Optional, auto-injected by system
) -> str:
"""Function description (shown to LLM)"""
return "result"
```
3. Restart the backend server
4. Reference in Agent node via `name: my_tool` or `name: my_module:All`

92
docs/user_guide/en/modules/tooling/mcp.md Executable file
View File

@ -0,0 +1,92 @@
# MCP Tooling Guide
MCP tooling is split into two explicit modes: **Remote (HTTP)** and **Local (stdio)**. They map to `tooling.type: mcp_remote` and `tooling.type: mcp_local`. The legacy `type: mcp` schema is no longer supported.
## 1. Mode overview
| Mode | Tooling type | When to use | Key fields |
| --- | --- | --- | --- |
| Remote | `mcp_remote` | A hosted HTTP(S) MCP server (FastMCP, Claude Desktop Connector, custom gateways) | `server`, `headers`, `timeout` |
| Local | `mcp_local` | A local executable that speaks MCP over stdio (Blender MCP, CLI tools, etc.) | `command`, `args`, `cwd`, `env`, timeouts |
## 2. `McpRemoteConfig`
| Field | Description |
| --- | --- |
| `server` | Required. MCP HTTP(S) endpoint, e.g. `https://api.example.com/mcp`. |
| `headers` | Optional. Extra HTTP headers such as `Authorization`. |
| `timeout` | Optional per-request timeout (seconds). |
**YAML example**
```yaml
nodes:
- id: remote_mcp
type: agent
config:
tooling:
type: mcp_remote
config:
server: https://mcp.mycompany.com/mcp
headers:
Authorization: Bearer ${MY_MCP_TOKEN}
timeout: 15
```
DevAll connects to the URL for each list/call request and passes `headers`. If the server is unreachable, an error is raised immediately—there is no local fallback.
## 3. `McpLocalConfig` fields
`mcp_local` declares the process arguments directly under `config`:
- `command` / `args`: executable and arguments (e.g., `uvx blender-mcp`).
- `cwd`: optional working directory.
- `env` / `inherit_env`: environment overrides.
- `startup_timeout`: max seconds to wait for `wait_for_log`.
- `wait_for_log`: regex matched against stdout to mark readiness.
**YAML example**
```yaml
nodes:
- id: local_mcp
type: agent
config:
tooling:
type: mcp_local
config:
command: uvx
args:
- blender-mcp
cwd: ${REPO_ROOT}
wait_for_log: "MCP ready"
startup_timeout: 8
```
DevAll keeps the process alive and relays MCP frames over stdio.
## 4. FastMCP sample server
`mcp_example/mcp_server.py`:
```python
from fastmcp import FastMCP
import random
mcp = FastMCP("Company Simple MCP Server", debug=True)
@mcp.tool
def rand_num(a: int, b: int) -> int:
return random.randint(a, b)
if __name__ == "__main__":
mcp.run()
```
Launch:
```bash
uv run fastmcp run mcp_example/mcp_server.py --transport streamable-http --port 8010
```
- Remote mode: set `server` to `http://127.0.0.1:8010/mcp`.
- Local mode: run the script with `transport=stdio` and point `command` to that invocation.
## 5. Security & operations
- **Network exposure**: Remote mode should sit behind HTTPS + ACL/API keys. Local mode still has access to the host filesystem, so keep the script sandboxed.
- **Resource cleanup**: Local mode processes are terminated by DevAll; make sure they gracefully handle SIGTERM/SIGKILL.
- **Logs**: Emit a clear readiness line that matches `wait_for_log` to debug startup issues.
- **Auth**: Remote mode handles tokens via `headers`; Local mode can receive secrets via `env` (never commit them).
- **Multi-session**: If the MCP server is single-tenant, cap concurrency (e.g., `max_concurrency=1`) and share the same YAML config.
## 6. Debugging checklist
1. Remote: ping the HTTP endpoint via curl or `fastmcp client`. Local: run the binary manually and confirm the readiness log.
2. Start DevAll (optionally with `--reload`) and observe backend logs for tool discovery.
3. When calls fail, inspect the Web UI tool traces or the structured logs under `logs/`.

152
docs/user_guide/en/nodes/agent.md Executable file
View File

@ -0,0 +1,152 @@
# Agent Node
The Agent node is the most fundamental node type in the DevAll platform, used to invoke Large Language Models (LLMs) for text generation, conversation, reasoning, and other tasks. It supports multiple model providers (OpenAI, Gemini, etc.) and can be configured with advanced features like tool calling, chain-of-thought, and memory.
## Configuration
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `provider` | string | Yes | `openai` | Model provider name, e.g., `openai`, `gemini` |
| `name` | string | Yes | - | Model name, e.g., `gpt-4o`, `gemini-2.0-flash-001` |
| `role` | text | No | - | System prompt |
| `base_url` | string | No | Provider default | API endpoint URL, supports `${VAR}` placeholders |
| `api_key` | string | No | - | API key, recommend using environment variable `${API_KEY}` |
| `params` | dict | No | `{}` | Model call parameters (temperature, top_p, etc.) |
| `tooling` | object | No | - | Tool calling configuration, see [Tooling Module](../modules/tooling/README.md) |
| `thinking` | object | No | - | Chain-of-thought configuration, e.g., chain-of-thought, reflection |
| `memories` | list | No | `[]` | Memory binding configuration, see [Memory Module](../modules/memory.md) |
| `retry` | object | No | - | Automatic retry strategy configuration |
### Retry Strategy Configuration (retry)
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `enabled` | bool | `true` | Whether to enable automatic retry |
| `max_attempts` | int | `5` | Maximum number of attempts (including first attempt) |
| `min_wait_seconds` | float | `1.0` | Minimum backoff wait time |
| `max_wait_seconds` | float | `6.0` | Maximum backoff wait time |
| `retry_on_status_codes` | list[int] | `[408,409,425,429,500,502,503,504]` | HTTP status codes that trigger retry |
## When to Use
- **Text generation**: Writing, translation, summarization, Q&A, etc.
- **Intelligent conversation**: Multi-turn dialogue, customer service bots
- **Tool calling**: Enable the model to call external APIs or execute functions
- **Complex reasoning**: Use with thinking configuration for deep thought
- **Knowledge retrieval**: Use with memories to implement RAG patterns
## Examples
### Basic Configuration
```yaml
nodes:
- id: Writer
type: agent
config:
provider: openai
base_url: ${BASE_URL}
api_key: ${API_KEY}
name: gpt-4o
role: |
You are a professional technical documentation writer. Please answer questions in clear and concise language.
params:
temperature: 0.7
max_tokens: 2000
```
### Configuring Tool Calling
```yaml
nodes:
- id: Assistant
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
tooling:
type: function # Tool type: function, mcp_remote, mcp_local
config:
tools: # List of function tools from functions/function_calling/ directory
- name: describe_available_files
- name: load_file
timeout: 20 # Optional: execution timeout (seconds)
```
### Configuring MCP Tools (Remote HTTP)
```yaml
nodes:
- id: MCP Agent
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
tooling:
type: mcp_remote
config:
server: http://localhost:8080/mcp # MCP server endpoint
headers: # Optional: custom request headers
Authorization: Bearer ${MCP_TOKEN}
timeout: 30 # Optional: request timeout (seconds)
```
### Configuring MCP Tools (Local stdio)
```yaml
nodes:
- id: Local MCP Agent
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
tooling:
type: mcp_local
config:
command: uvx # Launch command
args: ["mcp-server-sqlite", "--db-path", "data.db"]
cwd: ${WORKSPACE} # Optional, usually not needed
env: # Optional, usually not needed
DEBUG: "true"
startup_timeout: 10 # Optional: startup timeout (seconds)
```
### Gemini Multimodal Configuration
```yaml
nodes:
- id: Vision Agent
type: agent
config:
provider: gemini
base_url: https://generativelanguage.googleapis.com
api_key: ${GEMINI_API_KEY}
name: gemini-2.5-flash-image
role: You need to generate corresponding image content based on user input.
```
### Configuring Retry Strategy
```yaml
nodes:
- id: Robust Agent
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
retry: # Retry is enabled by default, you can customize it
enabled: true
max_attempts: 3
min_wait_seconds: 2.0
max_wait_seconds: 10.0
```
## Related Documentation
- [Tooling Module Configuration](../modules/tooling/README.md)
- [Memory Module Configuration](../modules/memory.md)
- [Workflow Authoring Guide](../workflow_authoring.md)

114
docs/user_guide/en/nodes/human.md Executable file
View File

@ -0,0 +1,114 @@
# Human Node
The Human node is used to introduce human interaction during workflow execution, allowing users to view the current state and provide input through the Web UI. This node blocks workflow execution until the user submits a response.
## Configuration
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `description` | text | No | - | Task description displayed to the user, explaining the operation that requires human completion |
## Core Concepts
### Blocking Wait Mechanism
When the workflow reaches a Human node:
1. The workflow pauses, waiting for human input
2. The Web UI displays the current context and task description
3. The user enters a response in the interface
4. The workflow continues execution, passing the user input to downstream nodes
### Web UI Interaction
- Human nodes are presented as a conversation in the Launch interface
- Users can view previous execution history
- Supports attachment uploads (e.g., files, images)
## When to Use
- **Review and confirmation**: Have humans review LLM output before continuing
- **Modification suggestions**: Collect user suggestions for modifying generated content
- **Critical decisions**: Points where human judgment is needed to continue
- **Data supplementation**: When additional information from the user is needed
- **Quality control**: Introduce human quality checks at critical nodes
## Examples
### Basic Configuration
```yaml
nodes:
- id: Human Reviewer
type: human
config:
description: Please review the above content. If satisfied, enter ACCEPT; otherwise, enter your modification suggestions.
```
### Human-Machine Collaboration Loop
```yaml
nodes:
- id: Article Writer
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
role: You are a professional writer who writes articles based on user requirements.
- id: Human Reviewer
type: human
config:
description: |
Please review the article:
- If satisfied with the result, enter ACCEPT to end the process
- Otherwise, enter modification suggestions to continue iterating
edges:
- from: Article Writer
to: Human Reviewer
- from: Human Reviewer
to: Article Writer
condition:
type: keyword
config:
none: [ACCEPT]
case_sensitive: false
```
### Multi-Stage Review
```yaml
nodes:
- id: Draft Generator
type: agent
config:
provider: openai
name: gpt-4o
- id: Content Review
type: human
config:
description: Please review content accuracy. Enter APPROVED or modification suggestions.
- id: Final Reviewer
type: human
config:
description: Final confirmation. Enter PUBLISH to publish or REJECT to reject.
edges:
- from: Draft Generator
to: Content Review
- from: Content Review
to: Final Reviewer
condition:
type: keyword
config:
any: [APPROVED]
```
## Best Practices
- Clearly explain expected operations and keywords in the `description`
- Use conditional edges with keywords to implement flow control
- Consider adding timeout mechanisms to avoid infinite workflow waiting

140
docs/user_guide/en/nodes/literal.md Executable file
View File

@ -0,0 +1,140 @@
# Literal Node
The Literal node is used to output fixed text content. When the node is triggered, it ignores all inputs and directly outputs a predefined message.
## Configuration
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `content` | text | Yes | - | Fixed text content to output, cannot be empty |
| `role` | string | No | `user` | Message role: `user` or `assistant` |
## Core Concepts
### Fixed Output
Characteristics of the Literal node:
- **Ignores input**: Regardless of what upstream content is passed in, it does not affect the output
- **Fixed content**: Outputs the same `content` every time it executes
- **Role marking**: Output message carries the specified role identifier
### Message Role
- `user`: Indicates this is a message sent by the user
- `assistant`: Indicates this is a message sent by the assistant (AI)
The role setting affects how downstream nodes process the message.
## When to Use
- **Fixed prompt injection**: Inject fixed instructions or context into the workflow
- **Testing and debugging**: Use fixed input to test downstream nodes
- **Default responses**: Return fixed messages under specific conditions
- **Process initialization**: Serve as the starting point of a workflow to provide initial content
## Examples
### Basic Usage
```yaml
nodes:
- id: Welcome Message
type: literal
config:
content: |
Welcome to the intelligent assistant! Please describe your needs.
role: assistant
```
### Injecting Fixed Context
```yaml
nodes:
- id: Context Injector
type: literal
config:
content: |
Please note the following rules:
1. Answers must be concise and clear
2. Reply in English
3. If uncertain, please state so
role: user
- id: Assistant
type: agent
config:
provider: openai
name: gpt-4o
edges:
- from: Context Injector
to: Assistant
```
### Fixed Responses in Conditional Branches
```yaml
nodes:
- id: Classifier
type: agent
config:
provider: openai
name: gpt-4o
role: Determine user intent, reply with KNOWN or UNKNOWN
- id: Known Response
type: literal
config:
content: I can help you complete this task.
role: assistant
- id: Unknown Response
type: literal
config:
content: Sorry, I cannot understand your request. Please describe it in a different way.
role: assistant
edges:
- from: Classifier
to: Known Response
condition:
type: keyword
config:
any: [KNOWN]
- from: Classifier
to: Unknown Response
condition:
type: keyword
config:
any: [UNKNOWN]
```
### Testing Purposes
```yaml
nodes:
- id: Test Input
type: literal
config:
content: |
This is a test text for verifying downstream processing logic.
Contains multiple lines.
role: user
- id: Processor
type: python
config:
timeout_seconds: 30
edges:
- from: Test Input
to: Processor
start: [Test Input]
```
## Notes
- The `content` field cannot be an empty string
- Use YAML multi-line string syntax `|` for writing long text
- Choose the correct `role` to ensure downstream nodes process the message correctly

153
docs/user_guide/en/nodes/loop_counter.md Executable file
View File

@ -0,0 +1,153 @@
# Loop Counter Node
The Loop Counter node is a loop control node used to limit the number of iterations of a loop in a workflow. Through a counting mechanism, it suppresses output before reaching the preset limit, and only releases the message to trigger outgoing edges when the limit is reached, thereby terminating the loop.
## Configuration
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `max_iterations` | int | Yes | `10` | Maximum number of loop iterations, must be ≥ 1 |
| `reset_on_emit` | bool | No | `true` | Whether to reset the counter after reaching the limit |
| `message` | text | No | - | Message content to send to downstream when limit is reached |
## Core Concepts
### How It Works
The Loop Counter node maintains an internal counter with the following behavior:
1. **Each time it is triggered**: Counter +1
2. **Counter < `max_iterations`**: **No output is produced**, outgoing edges are not triggered
3. **Counter = `max_iterations`**: Output message is produced, triggering outgoing edges
This "suppress-release" mechanism allows the Loop Counter to precisely control when a loop terminates.
### Topological Structure Requirements
The Loop Counter node has special placement requirements in the graph structure:
```
┌──────────────────────────────────────┐
▼ │
Agent ──► Human ─────► Loop Counter ──┬──┘
▲ │ │
└─────────┘ ▼
End Node (outside loop)
```
> **Important**: Since Loop Counter **produces no output until the limit is reached**:
> - **Human must connect to both Agent and Loop Counter**: This way the "continue loop" edge is handled by Human → Agent, while Loop Counter only handles counting
> - **Loop Counter must connect to Agent (inside loop)**: So it's recognized as an in-loop node, avoiding premature loop termination
> - **Loop Counter must connect to End Node (outside loop)**: When the limit is reached, trigger the out-of-loop node to terminate the entire loop execution
### Counter State
- Counter state persists throughout the entire workflow execution
- When `reset_on_emit: true`, the counter resets to 0 after reaching the limit
- When `reset_on_emit: false`, the counter continues accumulating after reaching the limit, outputting on every subsequent trigger
## When to Use
- **Preventing infinite loops**: Set a safety limit for human-machine interaction loops
- **Iteration control**: Limit the maximum number of self-improvement iterations for an Agent
- **Timeout protection**: Serve as a "circuit breaker" for process execution
## Examples
### Basic Usage
```yaml
nodes:
- id: Iteration Guard
type: loop_counter
config:
max_iterations: 5
reset_on_emit: true
message: Maximum iteration count reached, process terminated.
```
### Human-Machine Interaction Loop Protection
This is the most typical use case for Loop Counter:
```yaml
graph:
id: review_loop
description: Review loop with iteration limit
nodes:
- id: Writer
type: agent
config:
provider: openai
name: gpt-4o
role: Improve articles based on user feedback
- id: Reviewer
type: human
config:
description: |
Review the article, enter ACCEPT to accept or provide modification suggestions.
- id: Loop Guard
type: loop_counter
config:
max_iterations: 3
message: Maximum modification count (3 times) reached, process automatically ended.
- id: Final Output
type: passthrough
config: {}
edges:
# Main loop: Writer -> Reviewer
- from: Writer
to: Reviewer
# Condition 1: User enters ACCEPT -> End
- from: Reviewer
to: Final Output
condition:
type: keyword
config:
any: [ACCEPT]
# Condition 2: User enters modification suggestions -> Trigger both Writer to continue loop AND Loop Guard to count
- from: Reviewer
to: Writer
condition:
type: keyword
config:
none: [ACCEPT]
- from: Reviewer
to: Loop Guard
condition:
type: keyword
config:
none: [ACCEPT]
# Loop Guard connects to Writer (keeps it inside the loop)
- from: Loop Guard
to: Writer
# When Loop Guard reaches limit: Trigger Final Output to end the process
- from: Loop Guard
to: Final Output
start: [Writer]
end: [Final Output]
```
**Execution Flow Explanation**:
1. User first enters modification suggestions → Triggers both Writer (continue loop) and Loop Guard (count 1, no output)
2. User enters modification suggestions again → Triggers both Writer (continue loop) and Loop Guard (count 2, no output)
3. User enters modification suggestions for the third time → Writer continues execution, Loop Guard count 3 reaches limit, outputs message triggering Final Output, terminating the loop
4. Or at any time user enters ACCEPT → Goes directly to Final Output to end
## Notes
- `max_iterations` must be a positive integer (≥ 1)
- Loop Counter **produces no output until the limit is reached**, outgoing edges will not trigger
- Ensure Loop Counter connects to both in-loop and out-of-loop nodes
- The `message` field is optional, default message is `"Loop limit reached (N)"`

206
docs/user_guide/en/nodes/passthrough.md Executable file
View File

@ -0,0 +1,206 @@
# Passthrough Node
The Passthrough node is the simplest node type. It performs no operations and passes received messages to downstream nodes. By default, it only passes the **last message**. It is primarily used for graph structure "wire management" optimization and context control.
## Configuration
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `only_last_message` | bool | No | `true` | Whether to pass only the last message. Set to `false` to pass all messages. |
### Basic Configuration
```yaml
config: {} # Uses default configuration, only passes the last message
```
### Pass All Messages
```yaml
config:
only_last_message: false # Pass all received messages
```
## Core Concepts
### Pass-through Behavior
- Receives all messages passed from upstream
- **By default, only passes the last message** (`only_last_message: true`)
- When `only_last_message: false`, passes all messages
- Does not perform any content processing or transformation
### Graph Structure Optimization
The core value of the Passthrough node is not in data processing, but in **graph structure optimization** ("wire management"):
- Makes complex edge connections clearer
- Centrally manages outgoing edge configurations (such as `keep_message`)
- Serves as a logical dividing point, improving workflow readability
## Key Uses
### 1. As an Entry Node to Preserve Initial Context
Using Passthrough as the workflow entry node, combined with the `keep_message: true` edge configuration, ensures that the user's initial task is always preserved in the context and won't be overwritten by subsequent node outputs:
```yaml
nodes:
- id: Task Keeper
type: passthrough
config: {}
- id: Worker A
type: agent
config:
provider: openai
name: gpt-4o
- id: Worker B
type: agent
config:
provider: openai
name: gpt-4o
edges:
# Distribute tasks from entry, preserving original message
- from: Task Keeper
to: Worker A
keep_message: true # Preserve initial task context
- from: Task Keeper
to: Worker B
keep_message: true
start: [Task Keeper]
```
**Effect**: Both Worker A and Worker B can see the user's original input, not just the output from the previous node.
### 2. Filtering Redundant Output in Loops
In workflows containing loops, nodes within the loop may produce a large amount of intermediate output. Passing all outputs to subsequent nodes would cause context bloat. Using a Passthrough node allows you to **pass only the final result of the loop**:
```yaml
nodes:
- id: Iterative Improver
type: agent
config:
provider: openai
name: gpt-4o
role: Continuously improve output based on feedback
- id: Evaluator
type: agent
config:
provider: openai
name: gpt-4o
role: |
Evaluate output quality, reply GOOD or provide improvement suggestions
- id: Result Filter
type: passthrough
config: {}
- id: Final Processor
type: agent
config:
provider: openai
name: gpt-4o
role: Post-process the final result
edges:
- from: Iterative Improver
to: Evaluator
# Loop: Return to improvement node when evaluation fails
- from: Evaluator
to: Iterative Improver
condition:
type: keyword
config:
none: [GOOD]
# Loop ends: Filter through Passthrough, only pass the last message
- from: Evaluator
to: Result Filter
condition:
type: keyword
config:
any: [GOOD]
- from: Result Filter
to: Final Processor
start: [Iterative Improver]
end: [Final Processor]
```
**Effect**: Regardless of how many loop iterations occur, `Final Processor` will only receive the last output from `Evaluator` (the one indicating quality passed), not all intermediate results.
## Other Uses
- **Placeholder**: Reserve node positions during design phase
- **Conditional branching**: Implement routing logic with conditional edges
- **Debugging observation point**: Insert nodes for easy observation in the workflow
## Examples
### Basic Usage
```yaml
nodes:
- id: Router
type: passthrough
config: {}
```
### Conditional Routing
```yaml
nodes:
- id: Classifier
type: agent
config:
provider: openai
name: gpt-4o
role: |
Classify input content, reply TECHNICAL or BUSINESS
- id: Router
type: passthrough
config: {}
- id: Tech Handler
type: agent
config:
provider: openai
name: gpt-4o
- id: Biz Handler
type: agent
config:
provider: openai
name: gpt-4o
edges:
- from: Classifier
to: Router
- from: Router
to: Tech Handler
condition:
type: keyword
config:
any: [TECHNICAL]
- from: Router
to: Biz Handler
condition:
type: keyword
config:
any: [BUSINESS]
```
## Best Practices
- Use meaningful node IDs that describe their topological role (e.g., `Task Keeper`, `Result Filter`)
- When used as an entry node, configure outgoing edges with `keep_message: true` to preserve context
- Use after loops to filter out redundant intermediate output

89
docs/user_guide/en/nodes/python.md Executable file
View File

@ -0,0 +1,89 @@
# Python Node
The Python node is used to execute Python scripts or inline code within workflows, implementing custom data processing, API calls, file operations, and other logic. Scripts are executed in the shared `code_workspace/` directory and can access workflow context data.
## Configuration
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `interpreter` | string | No | Current Python | Python interpreter path |
| `args` | list[str] | No | `[]` | Startup arguments appended to the interpreter |
| `env` | dict[str, str] | No | `{}` | Additional environment variables, overrides system defaults |
| `timeout_seconds` | int | No | `60` | Script execution timeout (seconds) |
| `encoding` | string | No | `utf-8` | Encoding for parsing stdout/stderr |
## Core Concepts
### Code Workspace
Python scripts are executed within the `code_workspace/` directory:
- Scripts can read and write files in this directory
- Multiple Python nodes share the same workspace
- The workspace persists for the duration of a single workflow execution
### Input/Output
- **Input**: Outputs from upstream nodes are passed as environment variables or standard input
- **Output**: The script's stdout output will be passed as a Message to downstream nodes
## When to Use
- **Data processing**: Parse JSON/XML, data transformation, formatting
- **API calls**: Call third-party services, fetch external data
- **File operations**: Read/write files, generate reports
- **Complex calculations**: Mathematical operations, algorithm implementations
- **Glue logic**: Custom logic connecting different nodes
## Examples
### Basic Configuration
```yaml
nodes:
- id: Data Processor
type: python
config:
timeout_seconds: 120
env:
key: value
```
### Specifying Interpreter and Arguments
```yaml
nodes:
- id: Script Runner
type: python
config:
interpreter: /usr/bin/python3.11
timeout_seconds: 300
encoding: utf-8
```
### Typical Workflow Example
```yaml
nodes:
- id: LLM Generator
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
role: You need to generate executable Python code based on user input. The code should be wrapped in ```python ```.
- id: Result Parser
type: python
config:
timeout_seconds: 30
edges:
- from: LLM Generator
to: Result Parser
```
## Notes
- Ensure script files are placed in the `code_workspace/` directory
- Long-running scripts should have an appropriately increased `timeout_seconds`
- Use `env` to pass additional environment variables, accessible in scripts via `os.getenv`

138
docs/user_guide/en/nodes/subgraph.md Executable file
View File

@ -0,0 +1,138 @@
# Subgraph Node
The Subgraph node allows embedding another workflow graph into the current workflow, enabling process reuse and modular design. Subgraphs can come from external YAML files or be defined inline in the configuration.
## Configuration
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `type` | string | Yes | - | Subgraph source type: `file` or `config` |
| `config` | object | Yes | - | Contains different configurations depending on `type` |
### File Type Configuration
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `path` | string | Yes | Subgraph file path (relative to `yaml_instance/` or absolute path) |
### Config Type Configuration
Inline definition of the complete subgraph structure, containing the same fields as the top-level `graph`:
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `id` | string | Yes | Subgraph identifier |
| `description` | string | No | Subgraph description |
| `log_level` | string | No | Log level (DEBUG/INFO) |
| `nodes` | list | Yes | Node list |
| `edges` | list | No | Edge list |
| `start` | list | No | Entry node list |
| `end` | list | No | Exit node list |
| `memory` | list | No | Memory definitions specific to the subgraph |
## Core Concepts
### Modular Reuse
Extract commonly used process fragments into independent YAML files that can be reused by multiple workflows:
- Example: Package an "article polishing" process as a subgraph
- Different main workflows can call this subgraph
### Variable Inheritance
Subgraphs inherit `vars` variable definitions from the parent graph, supporting cross-level variable passing.
### Execution Isolation
Subgraphs execute as independent units with their own:
- Node namespace
- Log level configuration
- Memory definitions (optional)
## When to Use
- **Process reuse**: Multiple workflows sharing the same sub-processes
- **Modular design**: Breaking complex processes into manageable smaller units
- **Team collaboration**: Different teams maintaining different subgraph modules
## Examples
### Referencing External File
```yaml
nodes:
- id: Review Process
type: subgraph
config:
type: file
config:
path: common/review_flow.yaml
```
### Inline Subgraph Definition
```yaml
nodes:
- id: Translation Unit
type: subgraph
config:
type: config
config:
id: translation_subgraph
description: Multi-language translation subprocess
nodes:
- id: Translator
type: agent
config:
provider: openai
name: gpt-4o
role: You are a professional translator who translates content to the target language.
- id: Proofreader
type: agent
config:
provider: openai
name: gpt-4o
role: You are a proofreading expert who checks and polishes translated content.
edges:
- from: Translator
to: Proofreader
start: [Translator]
end: [Proofreader]
```
### Combining Multiple Subgraphs
```yaml
nodes:
- id: Input Handler
type: agent
config:
provider: openai
name: gpt-4o
- id: Analysis Module
type: subgraph
config:
type: file
config:
path: modules/analysis.yaml
- id: Report Module
type: subgraph
config:
type: file
config:
path: modules/report_gen.yaml
edges:
- from: Input Handler
to: Analysis Module
- from: Analysis Module
to: Report Module
```
## Notes
- Subgraph file paths support relative paths (based on `yaml_instance/`) and absolute paths
- Avoid circular nesting (A references B, B references A)
- The subgraph's `start` and `end` nodes determine how data flows in and out, which decides how the subgraph processes messages from the parent graph and which node's final output is returned to the parent graph

99
docs/user_guide/en/web_ui_guide.md Executable file
View File

@ -0,0 +1,99 @@
# Frontend Web UI Quick Start Guide
This guide helps users quickly get started with the DevAll Web UI, covering main functional pages and operation workflows.
## 1. System Entry
After starting frontend and backend services, visit `http://localhost:5173` to access the Web UI.
## 2. Main Pages
### 2.1 Home
System homepage providing quick navigation links.
### 2.2 Workflow List
View and manage all available workflow YAML files.
**Features**:
- Browse workflows in `yaml_instance/` directory
- Preview YAML configuration content
- Select workflow to execute or edit
### 2.3 Launch View
The main interface for workflow execution, the most commonly used page.
**Operation Flow**:
1. **Select workflow**: Choose YAML file from the left panel
2. **Upload attachments** (optional): Click upload button to add files (CSV data, images, etc.)
3. **Enter task prompt**: Input instructions in the text box to guide workflow execution
4. **Click Launch**: Start workflow execution
**During Execution**:
- **Nodes View**: Observe node status changes (pending → running → success/failed)
- **Output Panel**: View real-time execution logs, node output context, and generated artifacts (all share the same panel)
**Human Input**:
- When execution reaches a `human` node, the interface displays an input prompt
- Fill in text content or upload attachments, then submit to continue execution
### 2.4 Workflow Workbench
Visual workflow editor.
**Features**:
- Drag-and-drop node editing
- Node configuration panel
- Edge connections and condition settings
- Export to YAML file
### 2.5 Tutorial
Built-in tutorial to help new users understand system features.
## 3. Common Operations
### 3.1 Running a Workflow
1. Go to **Launch View**
2. Select workflow from the left panel
3. Enter Task Prompt
4. Click **Launch** button
5. Monitor execution progress and wait for completion
### 3.2 Downloading Results
After execution completes:
1. Click the **Download** button on the right panel
2. Download the complete Session archive (includes context.json, attachments, logs, etc.)
### 3.3 Human Review Node Interaction
When workflow contains `human` nodes:
1. Execution pauses and displays prompt message
2. Read context content
3. Enter review comments or action instructions
4. Click submit to continue execution
## 4. Keyboard Shortcuts
| Shortcut | Function |
|----------|----------|
| `Ctrl/Cmd + Enter` | Submit input |
| `Esc` | Close popup/panel |
## 5. Troubleshooting
| Issue | Solution |
|-------|----------|
| Page won't load | Confirm frontend `npm run dev` is running |
| Cannot connect to backend | Confirm backend `uv run python server_main.py` is running |
| Empty workflow list | Check `yaml_instance/` directory for YAML files |
| Execution unresponsive | Check browser DevTools Network/Console logs |
| WebSocket disconnected | Refresh page to re-establish connection |
## 6. Related Documentation
- [Workflow Authoring](workflow_authoring.md) - YAML writing guide

223
docs/user_guide/en/workflow_authoring.md Executable file
View File

@ -0,0 +1,223 @@
# Workflow Authoring Guide
This guide covers YAML structure, node types, provider configuration, edge conditions, and design template export so you can build and debug DevAll DAGs efficiently. Content mirrors `docs/user_guide/workflow_authoring.md` with English copy for global contributors.
## 1. Prerequisites
- Know the layout of `yaml_instance/` and `yaml_template/`.
- Understand the core node types (`model`, `python`, `agent`, `human`, `subgraph`, `passthrough`, `literal`).
- Review `FIELD_SPECS` (see [field_specs.md](field_specs.md)) and the Schema API contract ([config_schema_contract.md](config_schema_contract.md)) if you rely on dynamic forms in the frontend/IDE.
## 2. YAML Top-level Structure
Every workflow file follows the `DesignConfig` root with only three keys: `version`, `vars`, and `graph`. The snippet below is adapted from `yaml_instance/net_example.yaml` and can run as-is:
```yaml
version: 0.4.0
vars:
BASE_URL: https://api.example.com/v1
API_KEY: ${API_KEY}
graph:
id: paper_gen
description: Article generation and refinement
log_level: INFO
is_majority_voting: false
initial_instruction: |
Provide a word or short phrase and the workflow will draft and polish an article.
start:
- Article Writer
end:
- Article Writer
nodes:
- id: Article Writer
type: agent
config:
provider: openai
base_url: ${BASE_URL}
api_key: ${API_KEY}
name: gpt-4o
params:
temperature: 0.1
- id: Human Reviewer
type: human
config:
description: Review the article. Type ACCEPT to finish; otherwise provide revision notes.
edges:
- from: Article Writer
to: Human Reviewer
- from: Human Reviewer
to: Article Writer
condition:
type: keyword
config:
none:
- ACCEPT
case_sensitive: false
```
- `version`: optional configuration version (defaults to `0.0.0`). Increment it whenever schema changes in `entity/configs/graph.py` require template or migration updates.
- `vars`: root-level key-value map. You can reference `${VAR}` anywhere in the file; fallback is the same-name environment variable. `GraphDefinition.from_dict` rejects nested `vars`, so keep this block at the top only.
**Environment Variables & `.env` File**
The system supports referencing variables in YAML configurations using `${VAR}` syntax. These variables can be used in any string field within the configuration. Common use cases include:
- **API keys**: `api_key: ${API_KEY}`
- **Service URLs**: `base_url: ${BASE_URL}`
- **Model names**: `name: ${MODEL_NAME}`
The system automatically loads the `.env` file from the project root (if present) when parsing configurations. Variable resolution follows this priority order:
| Priority | Source | Description |
| --- | --- | --- |
| 1 (highest) | Values defined in `vars` | Key-value pairs declared directly in the YAML file |
| 2 | System/shell environment variables | Values set via `export` or system config |
| 3 (lowest) | Values from `.env` file | Only applied if the variable doesn't already exist |
> [!TIP]
> The `.env` file does not override existing environment variables. This allows you to define defaults in `.env` while overriding them via `export` or deployment platform configurations.
> [!WARNING]
> If a placeholder references a variable that is not defined in any of the three sources above, a `ConfigError` will be raised during configuration parsing with the exact path indicated.
- `graph`: required block that maps to the `GraphDefinition` dataclass:
- **Metadata**: `id` (required), `description`, `log_level` (default `DEBUG`), `is_majority_voting`, `initial_instruction`, and optional `organization`.
- **Execution controls**: `start`/`end` entry lists (the system executes nodes listed in `start` at the beginning), plus `nodes` and `edges`. Provider/model/tooling settings now live inside each `node.config`; the legacy top-level `providers` table is deprecated. In the example the `keyword` condition on `Human Reviewer -> Article Writer` keeps looping unless the reviewer types `ACCEPT`.
- **Shared resources**: `memory` defines stores available to `node.config.memories`. The validator ensures every attachment points to a declared store.
- **Schema references**: `yaml_template/design.yaml` mirrors the latest `GraphDefinition` shape. After editing configs run `python -m tools.export_design_template` or hit the Schema API to validate.
Further reading: `docs/user_guide/en/field_specs.md` (field catalog), `docs/user_guide/en/runtime_ops.md` (runtime observability), and `yaml_template/design.yaml` (generated baseline template).
## 3. Node Type Cheatsheet
| Type | Description | Key fields | Detailed Docs |
| --- |------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| --- | --- |
| `agent` | Runs an LLM-backed agent with optional tools, memories, and thinking phases. | `provider`, `model`, `prompt_template`, `tooling`, `thinking`, `memories` | [agent.md](nodes/agent.md) |
| `python` | Executes Python scripts/commands sharing the `code_workspace/`. | `entry_script`, `inline_code`, `timeout`, `env` | [python.md](nodes/python.md) |
| `human` | Pauses in the Web UI awaiting human input. | `prompt`, `timeout`, `attachments` | [human.md](nodes/human.md) |
| `subgraph` | Embeds a child DAG to reuse complex flows. | `graph_path` or inline `graph` | [subgraph.md](nodes/subgraph.md) |
| `passthrough` | Pass-through node that forwards only the last message by default and can be configured to forward all messages; used for context filtering and graph structure optimization. | `only_last_message` | [passthrough.md](nodes/passthrough.md) |
| `literal` | Emits a fixed text payload whenever triggered and discards inputs. | `content`, `role` (`user`/`assistant`) | [literal.md](nodes/literal.md) |
| `loop_counter` | Guard node that limits loop iterations before releasing downstream edges. | `max_iterations`, `reset_on_emit`, `message` | [loop_counter.md](nodes/loop_counter.md) |
Fetch the full schema via `POST /api/config/schema` or inspect the dataclasses inside `entity/configs/`.
## 4. Providers & Agent Settings
- When a node omits `provider`, the engine uses `globals.default_provider` (e.g., `openai`).
- Fields such as `model`, `api_key`, and `base_url` accept `${VAR}` placeholders for environment portability.
- When combining multiple providers, define `globals` in the workflow root (`{ default_provider: ..., retry: {...} }`) if supported by the dataclass.
### 4.1 Gemini Provider Config Example
```yaml
model:
provider: gemini
base_url: https://generativelanguage.googleapis.com
api_key: ${GEMINI_API_KEY}
name: gemini-2.0-flash-001
input_mode: messages
params:
response_modalities: ["text", "image"]
safety_settings:
- category: HARM_CATEGORY_SEXUAL
threshold: BLOCK_LOWER
```
The Gemini Provider supports multi-modal input (images/video/audio are automatically converted to Parts) and supports `function_calling_config` to control tool execution behavior.
## 5. Edges & Conditions
- Basic edge:
```yaml
- source: plan
target: execute
```
- Conditional edge:
```yaml
edges:
- source: router
target: analyze
condition: should_analyze # functions/edge/should_analyze.py
```
- If a `condition` function raises, the scheduler marks the branch as failed and stops downstream execution.
### 5.1 Edge Payload Processors
- Add `process` to an edge when you want to transform or filter the payload after the condition is met (e.g., extract a verdict, keep only structured fields, or rewrite text).
- Structure mirrors `condition` (`type + config`). Built-ins include:
- `regex_extract`: Python regex with optional `group`, `mode` (`replace_content`, `metadata`, `data_block`), `multiple`, and `on_no_match` (`pass`, `default`, `drop`).
- `function`: calls helpers under `functions/edge_processor/`. The handler signature is `def foo(payload: Message, **kwargs) -> Message | None`. Note: The Processor interface is now standardized, and `kwargs` includes `context: ExecutionContext`, allowing access to the current execution context.
- Example:
```yaml
- from: reviewer
to: qa
process:
type: regex_extract
config:
pattern: "Score\\s*:\\s*(?P<score>\\d+)"
group: score
mode: metadata
metadata_key: quality_score
case_sensitive: false
on_no_match: default
default_value: "0"
```
## 6. Agent Node Advanced Features
- **Tooling**: Configure `AgentConfig.tooling`; see the [Tooling module](modules/tooling/README.md) (Chinese for now).
- **Thinking**: Enable staged reasoning via `AgentConfig.thinking` (e.g., chain-of-thought, reflection). Reference `entity/configs/thinking.py` for parameters.
- **Memories**: Attach `MemoryAttachmentConfig` through `AgentConfig.memories`; details live in the [Memory module](modules/memory.md).
## 7. Dynamic Execution (Map-Reduce/Tree)
Nodes support a sibling field `dynamic` to enable parallel processing or Map-Reduce patterns.
### 7.1 Core Concepts
- **Map Mode** (`type: map`): Fan-out. Splits list inputs into multiple units for parallel execution, outputting `List[Message]` (flattened results).
- **Tree Mode** (`type: tree`): Fan-out & Reduce. Splits inputs for parallel execution, then recursively reduces results in groups of `group_size` until a single result remains (e.g., "summary of summaries").
- **Split Strategy**: Defines how to partition the output of the previous node or current input into parallel units.
### 7.2 Configuration Structure
```yaml
nodes:
- id: Research Agents
type: agent
# Standard config (behaves as template for parallel units)
config:
provider: openai
model: gpt-4o
prompt_template: "Research this topic: {{content}}"
# Dynamic execution config
dynamic:
type: map
# Split strategy (first layer only)
split:
type: message # Options: message, regex, json_path
# pattern: "..." # Required for regex mode
# json_path: "$.items[*]" # Required for json_path mode
# Mode-specific config
config:
max_parallel: 5 # Concurrency limit
```
### 7.3 Tree Mode Example
Ideal for chunked summarization of long texts:
```yaml
dynamic:
type: tree
split:
type: regex
pattern: "(?s).{1,2000}(?:\\s|$)" # Split every ~2000 chars
config:
group_size: 3 # Reduce every 3 results into 1
max_parallel: 10
```
This mode automatically builds a multi-level execution tree until the result count is reduced to 1. Split config is the same as map mode.
## 8. Design Template Export
After editing configs or `FIELD_SPECS`, regenerate templates:
```bash
python -m tools.export_design_template \
--output yaml_template/design.yaml \
--mirror frontend/public/design_0.4.0.yaml
```
- The script scans registered nodes, memories, tooling, and `FIELD_SPECS` to emit YAML templates plus the frontend mirror file.
- Commit the generated files and notify frontend owners to refresh static assets.
## 9. CLI / API Execution Paths
- **Web UI**: Choose a YAML file, fill run parameters, start execution, and monitor in the dashboard. *Recommended path.*
- **HTTP**: `POST /api/workflow/execute` with `session_name`, `graph_path` or `graph_content`, `task_prompt`, optional `attachments`, and `log_level` (defaults to `INFO`, supports `INFO` or `DEBUG`).
- **CLI**: `python run.py --path yaml_instance/demo.yaml --name test_run`. Provide `TASK_PROMPT` via env var or respond to the CLI prompt.
## 10. Debugging Tips
- Use the Web UI context snapshots or `WareHouse/<session>/context.json` to inspect node I/O. Note that all node outputs are now standardized as `List[Message]`.
- Leverage the Schema API breadcrumbs ([config_schema_contract.md](config_schema_contract.md)) or run `python run.py --inspect-schema` to view field specs quickly.
- Missing YAML placeholders trigger `ConfigError` during parsing with a precise path surfaced in both UI and CLI logs.

213
docs/user_guide/en/ws_frontend_logic.md Executable file
View File

@ -0,0 +1,213 @@
# WebSocket Connection Lifecycle Analysis
## Scenario: Workflow Completed/Cancelled → File Change/Relaunch → Launch
### Initial State (Workflow Completed/Cancelled)
**Variables:**
- `status.value = 'Completed'` or `'Cancelled'`
- `isWorkflowRunning.value = false`
- `shouldGlow.value = true` (set by watch on status)
- `ws` = existing WebSocket connection (still open)
- `sessionId` = current session ID
- `isConnectionReady.value = true`
---
## Scenario 1: Another File is Chosen
### Step 1: File Selection Triggers Watch
**Function:** `watch(selectedFile, (newFile) => {...})` (line 879)
**Process:**
1. `taskPrompt.value = ''` - clears input
2. `fileSearchQuery.value = newFile || ''` - updates search query
3. `isFileSearchDirty.value = false` - resets search state
### Step 2: WebSocket Disconnection
**Function:** `resetConnectionState({ closeSocket: true })` (line 443)
**Called at:** line 891
**What happens:**
- ✅ **WebSocket DISCONNECTED** - `ws.close()` is called (line 446)
- `ws = null` (line 452)
- `sessionId = null` (line 453)
- `isConnectionReady.value = false` (line 454)
- `shouldGlow.value = false` (line 455)
- `isWorkflowRunning.value = false` (line 456)
- `activeNodes.value = []` (line 457)
- Clears attachment timeouts and uploaded attachments
**Status:** `status.value = 'Connecting...'` (line 892)
### Step 3: Load YAML
**Function:** `handleYAMLSelection(newFile)` (line 706)
**Called at:** line 893
**What happens:**
- Clears `chatMessages.value = []`
- Fetches YAML file content
- Parses YAML and stores in `workflowYaml.value`
- Displays `initial_instruction` as notification
- Loads VueFlow graph
### Step 4: WebSocket Reconnection
**Function:** `establishWebSocketConnection()` (line 807)
**Called at:** line 894
**What happens:**
1. **Double Reset:** Calls `resetConnectionState()` again (line 809) - ensures clean state
2. **New WebSocket Created:**
- `const socket = new WebSocket('ws://localhost:8000/ws')` (line 816)
- `ws = socket` (line 817)
3. **Connection Events:**
- `socket.onopen` (line 819): Logs "WebSocket connected"
- `socket.onmessage` (line 825):
- Receives `connection` message with `session_id`
- Sets `sessionId` (line 832)
- Sets `isConnectionReady.value = true` (line 842)
- Sets `shouldGlow.value = true` (line 843)
- Sets `status.value = 'Waiting for launch...'` (line 844)
- `socket.onerror` (line 854): Handles connection errors
- `socket.onclose` (line 864): Handles disconnection
**Status:** `status.value = 'Waiting for launch...'` (after connection message received)
---
## Scenario 2: Relaunch Button Clicked
### Step 1: Button Click Handler
**Function:** `handleButtonClick()` (line 740)
**Triggered:** When Launch button is clicked and status is 'Completed'/'Cancelled'
**Condition Check:**
```javascript
else if (status.value === 'Completed' || status.value === 'Cancelled')
```
### Step 2: WebSocket Disconnection
**Function:** `resetConnectionState()` (line 443)
**Called at:** line 754
**What happens:**
- ✅ **WebSocket DISCONNECTED** - `ws.close()` is called
- All state variables reset (same as Scenario 1, Step 2)
**Status:** `status.value = 'Connecting...'` (line 755)
### Step 3: Load YAML
**Function:** `handleYAMLSelection(selectedFile.value)` (line 706)
**Called at:** line 756
**What happens:** Same as Scenario 1, Step 3
### Step 4: WebSocket Reconnection
**Function:** `establishWebSocketConnection()` (line 807)
**Called at:** line 757
**What happens:** Same as Scenario 1, Step 4
**Status:** `status.value = 'Waiting for launch...'` (after connection message received)
---
## Step 5: Launch Button Clicked (After Reconnection)
### Launch Workflow
**Function:** `launchWorkflow()` (line 1124)
**Triggered:** When Launch button is clicked and status is 'Waiting for launch...'
**Prerequisites Check:**
- `selectedFile.value` must exist
- `taskPrompt.value.trim()` or `attachmentIds.length > 0` must exist
- `ws`, `isConnectionReady.value`, and `sessionId` must be valid
**What happens:**
1. Sets `shouldGlow.value = false` (line 1150)
2. Sets `status.value = 'Launching...'` (line 1151)
3. Sends POST request to `/api/workflow/execute` with:
- `yaml_file`: selected file name
- `task_prompt`: user input
- `session_id`: current session ID
- `attachments`: uploaded attachment IDs
4. On success:
- Clears uploaded attachments
- Adds user dialogue to chat
- Sets `status.value = 'Running...'` (line 1185)
- Sets `isWorkflowRunning.value = true` (line 1186)
**Status:** `status.value = 'Running...'`
---
## Key Variables and Their Roles
### WebSocket State Variables
| Variable | Type | Purpose | Changes When |
|----------|------|---------|--------------|
| `ws` | `WebSocket \| null` | Current WebSocket connection | Set to `null` on disconnect, new socket on connect |
| `sessionId` | `string \| null` | Current session identifier | Set from connection message, cleared on reset |
| `isConnectionReady` | `ref<boolean>` | Whether connection is ready for launch | `true` after connection message, `false` on reset |
### Status Variables
| Variable | Type | Purpose | Values |
|----------|------|---------|--------|
| `status` | `ref<string>` | Current workflow status | 'Completed', 'Cancelled', 'Connecting...', 'Waiting for launch...', 'Launching...', 'Running...' |
| `isWorkflowRunning` | `ref<boolean>` | Whether workflow is actively running | `true` during execution, `false` otherwise |
| `shouldGlow` | `ref<boolean>` | UI glow effect state | `true` when ready for input, `false` during execution |
### Workflow State Variables
| Variable | Type | Purpose |
|----------|------|---------|
| `selectedFile` | `ref<string>` | Currently selected YAML file |
| `workflowYaml` | `ref<object>` | Parsed YAML content |
| `activeNodes` | `ref<string[]>` | List of currently active node IDs |
| `chatMessages` | `ref<array>` | All chat messages and notifications |
---
## WebSocket Connection Timeline
### When Disconnected:
1. **File Change:** `watch(selectedFile)` → `resetConnectionState()` → `ws.close()`
2. **Relaunch:** `handleButtonClick()` → `resetConnectionState()` → `ws.close()`
3. **Error/Close Events:** `socket.onerror` or `socket.onclose` → `resetConnectionState({ closeSocket: false })`
### When Reconnected:
1. **File Change:** `watch(selectedFile)` → `establishWebSocketConnection()` → `new WebSocket()`
2. **Relaunch:** `handleButtonClick()` → `establishWebSocketConnection()` → `new WebSocket()`
### Connection States:
```
[Completed/Cancelled]
↓ (File Change or Relaunch)
[Disconnected] → resetConnectionState() closes ws
↓
[Connecting...] → establishWebSocketConnection() creates new ws
↓
[WebSocket.onopen] → socket opened
↓
[WebSocket.onmessage: 'connection'] → sessionId received
↓
[Waiting for launch...] → isConnectionReady = true
↓ (Launch clicked)
[Launching...] → POST /api/workflow/execute
↓
[Running...] → isWorkflowRunning = true
```
---
## Important Notes
1. **Double Reset Issue:** `establishWebSocketConnection()` calls `resetConnectionState()` at the start (line 809), which means when called after `resetConnectionState()` in the caller, it's resetting twice. This is safe but redundant.
2. **Stale Socket Protection:** All WebSocket event handlers check `if (ws !== socket) return` to ignore events from old sockets that were replaced.
3. **Status Transitions:** The status goes through these states:
- `Completed/Cancelled` → `Connecting...` → `Waiting for launch...` → `Launching...` → `Running...`
4. **Connection Ready Check:** `launchWorkflow()` verifies `ws`, `isConnectionReady.value`, and `sessionId` before allowing launch.
5. **Automatic Reconnection:** Both file change and relaunch automatically establish a new WebSocket connection - no manual reconnection needed.

84
docs/user_guide/zh/attachments.md Executable file
View File

@ -0,0 +1,84 @@
# 附件与工件 API 指南
**说明**：此文档面向高级用户，一般场景下无需直接调用附件/工件 API，前端会自行处理。
附件（Attachment）是 Session 生命周期内可上传、下载、由节点注册的文件；工件（Artifact）是对附件事件的抽象，用于实时监听。本文档汇总 REST/WS 接口及存储策略，填补旧版 `frontend_attachment_api.md` 的缺口。
## 1. 上传与列举
### 1.1 上传文件
`POST /api/uploads/{session_id}`
- **Headers**：`Content-Type: multipart/form-data`
- **Form 字段**：`file`（单个文件）。
- **响应**：
```json
{
"attachment_id": "att_bxabcd",
"name": "spec.md",
"mime": "text/markdown",
"size": 12345
}
```
- 文件保存到 `WareHouse/<session>/code_workspace/attachments/`，并记录在 `attachments_manifest.json`。
### 1.2 列举附件
`GET /api/uploads/{session_id}`
- 返回该 Session 当前所有附件的元数据（ID、文件名、mime、大小、来源）。
### 1.3 在执行请求中引用
- `POST /api/workflow/execute` 或 WebSocket `human_input` 消息中可带 `attachments: ["att_xxx"]`，并必须同时提供 `task_prompt`（即便只想上传文件）。
## 2. 工件事件与下载
### 2.1 实时事件
`GET /api/sessions/{session_id}/artifact-events`
- Query：`after`, `wait_seconds`, `include_mime`, `include_ext`, `max_size`, `limit`。
- 响应含 `events[]`, `next_cursor`, `has_more`, `timed_out`。
- 每条事件：
```json
{
"artifact_id": "art_123",
"attachment_id": "att_456",
"node_id": "python_runner",
"path": "code_workspace/result.json",
"size": 2048,
"mime": "application/json",
"hash": "sha256:...",
"timestamp": 1732699900
}
```
- WebSocket 会镜像此事件（类型 `artifact_created`），前端可直接订阅。
### 2.2 下载单个工件
`GET /api/sessions/{session_id}/artifacts/{artifact_id}`
- Query：`mode=meta|stream`, `download=true|false`。
- **meta**：仅返回元数据。
- **stream**：返回文件内容；`download=true` 时附带 `Content-Disposition`。
- 小文件可选择 `data_uri` 内联（若服务器启用）。
### 2.3 打包下载 Session
`GET /api/sessions/{session_id}/download`
- 将 `WareHouse/<session>/` 打包为 zip，供一次性下载。
## 3. 文件生命周期
1. 上传：写入 `code_workspace/attachments/`，manifest 记录 `source`、`workspace_path`、`storage` 等字段。
2. Python 节点或工具可调用 `AttachmentStore.register_file()` 把 workspace 文件注册为附件；`WorkspaceArtifactHook` 会将其同步到事件流。
3. 默认保留所有附件，便于运行结束后下载。如果希望自动清理，设置 `MAC_AUTO_CLEAN_ATTACHMENTS=1`（只在 Session 完成后删除 `attachments/` 目录）。
4. WareHouse 打包下载不会删除原文件，需要额外策略（cron/job）做归档或清空。
## 4. 大小与安全建议
- **大小限制**：后端未硬编码，可在反向代理设置 `client_max_body_size`、`max_request_body_size`，或在自定义分支的 `AttachmentService.save_upload_file` 中添加校验。
- **文件类型**：基于 MIME 推断 `MessageBlockType`（image/audio/video/file）；可结合 `include_mime` 过滤。
- **病毒/敏感信息**：上传前由客户端自查；必要时在保存后触发扫描服务。
- **权限**：Attachment API 依赖 Session ID；生产部署应在代理层或 JWT 内部校验调用者身份，避免越权下载。
## 5. 常见问题
| 问题 | 排查步骤 |
| --- | --- |
| 上传 413/413 Payload Too Large | 调整反向代理或 FastAPI `client_max_size`，确认磁盘配额 |
| 下载链接 404 | 确认 `session_id` 拼写（仅允许字母/数字/`_-`），检查 Session 是否已被清理 |
| 工件事件缺失 | 确认 WebSocket 是否连接，或在 REST 事件接口中使用 `after` 游标重拉 |
| 附件未在 Python 节点可见 | 检查 `code_workspace/attachments/` 是否被清理、或 `_context['python_workspace_root']` 是否正确 |
## 6. 客户端实现建议
- Web UI：使用 `artifact-events` 长轮询或 WebSocket，实时刷新附件列表；在节点成功后提供“下载全部”按钮。
- CLI/自动化：在运行结束后调用 `/download` 拉取 zip；若仅需部分文件，可结合 `artifact-events` 的 `include_ext` 精准过滤。
- 测试环境：可通过脚本模拟上传/下载流程，确保反向代理和 CORS 配置正确。

95
docs/user_guide/zh/config_schema_contract.md Executable file
View File

@ -0,0 +1,95 @@
# 配置 Schema API 契约
本参考说明 `/api/config/schema` 与 `/api/config/schema/validate` 如何暴露 DevAll 的动态配置元数据，便于前端表单、IDE/CLI 通过 breadcrumbs（路径面包屑）按需获取局部 Schema。
## 1. 接口
| 方法 | 作用 |
| --- | --- |
| `POST /api/config/schema` | 根据 breadcrumbs 返回对应配置节点的字段定义。 |
| `POST /api/config/schema/validate` | 校验一份 YAML/JSON 文档，并可回传局部 Schema。 |
### 1.1 请求体（公共字段）
```json
{
"breadcrumbs": [
{"node": "DesignConfig", "field": "graph"},
{"node": "GraphConfig", "field": "nodes"},
{"node": "NodeConfig", "value": "model"}
]
}
```
- `node`（必填）：当前所处的类名（如 `DesignConfig`、`GraphConfig`、`NodeConfig`）。
- `field`（可选）：要下钻的子字段名；缺省表示仅断言仍在该 `node`。
- `value`（可选）：当子类由判别字段决定时填写（如节点 `type`）。值与 YAML 中保持一致。
- `index`（可选 int）：预留用于列表遍历，当前以 `field`/`value` 为主。
### 1.2 `/schema` 响应示例
```json
{
"schemaVersion": "0.1.0",
"node": "NodeConfig",
"fields": [
{"name": "id", "typeHint": "str", "required": true, "description": "Unique node identifier"},
{"name": "type", "typeHint": "str", "required": true,
"enum": ["model","python","agent"],
"enumOptions": [{"value":"model","label":"LLM Node","description":"Runs provider-backed models"}]
}
],
"constraints": [...],
"breadcrumbs": [...],
"cacheKey": "f90d..."
}
```
- `fields`：序列化的 `ConfigFieldSpec`；若有子配置，会包含 `childRoutes`。
- `constraints`：由 `collect_schema()` 生成的互斥/组合约束。
- `cacheKey`：基于 `{node, breadcrumbs}` 的 SHA-1，可用于客户端缓存。
### 1.3 `/schema/validate` 额外字段
请求体在 breadcrumbs 旁加入 `document`：
```json
{
"breadcrumbs": [{"node": "DesignConfig"}],
"document": "name: demo\nversion: 0.4.0\nworkflow:\n nodes: []\n edges: []\n"
}
```
响应：
- 通过：`{ "valid": true, "schema": { ... } }`
- 配置错误：
```json
{
"valid": false,
"error": "field 'nodes' must not be empty",
"path": ["workflow","nodes"],
"schema": { ... }
}
```
- YAML 解析失败：HTTP 400，payload `{ "message": "invalid_yaml", "error": "..." }`
## 2. Breadcrumb 使用提示
- 起点：`{ "node": "DesignConfig" }`。
- 每一步的 `node` 必须与当前位置的类匹配，否则返回 422。
- 用 `field` 进入子配置（graph → nodes → config 等）。
- 判别式子类（如节点 `type`、tooling `type`）需填写 `value`。
- 不可导航的字段会返回 `field '<name>' on <node> is not navigable`。
## 3. CLI 辅助
```bash
python run.py --inspect-schema --schema-breadcrumbs '[{"node":"DesignConfig","field":"graph"}]'
```
输出与 `/schema` 相同，便于在导出模板前调试 `FIELD_SPECS` 或注册表。
## 4. 前端调用范式
1. 以 `[{node:'DesignConfig', field:'graph'}]` 拉取基础表单。
2. 用户展开子配置（节点、tooling 等）时，附加相应 breadcrumbs 再取一次 Schema。
3. 用 `cacheKey + breadcrumbs` 做客户端缓存。
4. 保存前调用 `/schema/validate`，将 `error` + `path` 显示在表单中。
## 5. 错误参考
| HTTP | 场景 | Payload |
| --- | --- | --- |
| 400 | YAML 解析失败 | `{ "message": "invalid_yaml", "error": "..." }` |
| 422 | Breadcrumb 解析失败 | `{ "message": "breadcrumb node 'X'..." }` |
| 200 + `valid=false` | 后端 `ConfigError` | `{ "error": "...", "path": ["workflow", ...] }` |
| 200 + `valid=true` | 文档有效 | 返回所请求的 Schema，便于表单渲染。 |
搭配 `FIELD_SPECS` 使用，可在前端/IDE 构建无需硬编码的配置体验。

276
docs/user_guide/zh/dynamic_execution.md Executable file
View File

@ -0,0 +1,276 @@
# Dynamic 执行模式指南
Dynamic 执行模式允许在边级别定义并行处理行为，支持 Map（扇出）和 Tree（扇出+归约）两种模式。当消息通过配置了 `dynamic` 的边传递时，目标节点会根据拆分结果动态扩展为多个并行实例。
## 1. 概述
| 模式 | 描述 | 输出 | 适用场景 |
|------|------|------|----------|
| **Map** | 扇出执行，将消息拆分为多个单元并行处理 | `List[Message]`（打平结果） | 批量处理、并行查询 |
| **Tree** | 扇出+归约，并行处理后按组递归合并 | 单个 `Message` | 长文本摘要、层级聚合 |
## 2. 配置结构
Dynamic 配置定义在**边**上，而非节点：
```yaml
edges:
- from: Source Node
to: Target Node
trigger: true
carry_data: true
dynamic: # 边级动态执行配置
type: map # map 或 tree
split: # 消息拆分策略
type: message # message | regex | json_path
# pattern: "..." # regex 模式必填
# json_path: "..." # json_path 模式必填
config: # 模式特定配置
max_parallel: 5 # 最大并发数
```
### 2.1 核心概念
- **动态边**：配置了 `dynamic` 的边，其传递的消息会触发目标节点的动态扩展
- **静态边**：未配置 `dynamic` 的边，其传递的消息会**复制**到所有动态扩展实例
- **目标节点扩展**：目标节点根据 split 结果被"虚拟"扩展为多个并行实例
### 2.2 多入边一致性规则
> [!IMPORTANT]
> 当一个节点有多条入边配置了 `dynamic` 时，所有动态边的配置**必须完全一致**（type、split、config），否则执行时会报错。
## 3. Split 拆分策略
Split 定义如何将通过边的消息拆分为并行执行单元。
### 3.1 message 模式（默认）
每条通过边的消息作为独立执行单元。这是最常用的模式。
```yaml
split:
type: message
```
**执行行为**：
- 源节点输出 4 条消息通过动态边
- 拆分为 4 个并行单元，目标节点执行 4 次
### 3.2 regex 模式
使用正则表达式从文本内容中提取匹配项。
```yaml
split:
type: regex
pattern: "(?s).{1,2000}(?:\\s|$)" # 每 2000 字符切分
```
**典型用例**：
- 按段落拆分：`pattern: "\\n\\n"`
- 按行拆分：`pattern: ".+"`
- 按固定长度：`pattern: "(?s).{1,N}"`
### 3.3 json_path 模式
从 JSON 格式输出中按路径提取数组元素。
```yaml
split:
type: json_path
json_path: "$.items[*]" # JSONPath 表达式
```
## 4. Map 模式详解
Map 模式将消息拆分后并行执行目标节点，输出结果打平为 `List[Message]`。
### 4.1 配置项
| 字段 | 类型 | 默认值 | 说明 |
|------|------|--------|------|
| `max_parallel` | int | 10 | 最大并发执行数 |
### 4.2 执行流程
```mermaid
flowchart LR
Source["源节点输出"] --> Edge["动态边 (map)"]
Edge --> Split["拆分"]
Split --> U1["单元 1"]
Split --> U2["单元 2"]
Split --> U3["单元 N"]
U1 --> P1["目标节点 #1"]
U2 --> P2["目标节点 #2"]
U3 --> P3["目标节点 #N"]
P1 --> Merge["合并结果"]
P2 --> Merge
P3 --> Merge
Merge --> Output["List[Message]"]
```
## 5. Tree 模式详解
Tree 模式在 Map 基础上增加归约层，将并行结果按组递归合并，最终输出单个结果。
### 5.1 配置项
| 字段 | 类型 | 默认值 | 说明 |
|------|------|--------|------|
| `group_size` | int | 3 | 每组归约的元素数量，最小为 2 |
| `max_parallel` | int | 10 | 每层最大并发执行数 |
### 5.2 执行流程
```mermaid
flowchart TB
subgraph Layer1["第一层：并行执行"]
I1["单元 1"] --> R1["结果 1"]
I2["单元 2"] --> R2["结果 2"]
I3["单元 3"] --> R3["结果 3"]
I4["单元 4"] --> R4["结果 4"]
I5["单元 5"] --> R5["结果 5"]
I6["单元 6"] --> R6["结果 6"]
end
subgraph Layer2["第二层：分组归约 (group_size=3)"]
R1 & R2 & R3 --> G1["归约组 1"]
R4 & R5 & R6 --> G2["归约组 2"]
end
subgraph Layer3["第三层：最终归约"]
G1 & G2 --> Final["最终结果"]
end
```
## 6. 静态边消息复制
当目标节点同时有动态入边和静态入边时：
- **动态边消息**：按 split 策略拆分，每个单元执行一次目标节点
- **静态边消息**：**复制**到每个动态扩展实例
```yaml
nodes:
- id: Task Generator
type: passthrough
config: ...
- id: Extra Requirement
type: literal
config:
content: "请使用简洁的语言"
- id: Processor
type: agent
config:
name: gpt-4o
role: 处理任务
edges:
- from: Task Generator
to: Processor
dynamic: # 动态边：4 条任务 → 4 个并行单元
type: map
split:
type: message
config:
max_parallel: 10
- from: Extra Requirement
to: Processor # 静态边：复制到所有 4 个实例
trigger: true
carry_data: true
```
**执行结果**：Processor 执行 4 次，每次收到 1 条任务 + "请使用简洁的语言"
## 7. 完整示例
### 7.1 旅行规划（Map + Tree 组合）
```yaml
graph:
nodes:
- id: Eat Planner
type: literal
config:
content: 请规划在上海吃什么
role: user
- id: Play Planner
type: literal
config:
content: 请规划在上海玩什么
role: user
- id: Stay Planner
type: literal
config:
content: 请规划在上海住哪里
role: user
- id: Collector
type: passthrough
config:
only_last_message: false
- id: Travel Executor
type: agent
config:
name: gpt-4o
role: 你是旅行规划师，请按照用户请求进行规划
- id: Final Aggregator
type: agent
config:
name: gpt-4o
role: 请将输入的内容整合成一份完整的旅行计划
edges:
- from: Eat Planner
to: Collector
- from: Play Planner
to: Collector
- from: Stay Planner
to: Collector
- from: Collector
to: Travel Executor
dynamic: # Map 扇出：3 个规划请求 → 3 个并行执行
type: map
split:
type: message
config:
max_parallel: 10
- from: Travel Executor
to: Final Aggregator
dynamic: # Tree 归约：3 个结果 → 1 个最终计划
type: tree
split:
type: message
config:
group_size: 2
max_parallel: 10
```
### 7.2 长文档摘要（Tree 模式）
```yaml
edges:
- from: Document Source
to: Summarizer
dynamic:
type: tree
split:
type: regex
pattern: "(?s).{1,2000}(?:\\s|$)" # 2000 字符切分
config:
group_size: 3
max_parallel: 10
```
## 8. 性能建议
- **控制并发**：设置合理的 `max_parallel` 避免触发 API 限流
- **优化拆分粒度**：过细的拆分增加开销，过粗则无法充分并行
- **Tree 组大小**：`group_size=2-4` 通常是较好的选择
- **监控成本**：Dynamic 模式会显著增加 API 调用次数
## 9. 相关文档
- [边配置指南](../edges.md)
- [工作流编排指南](../workflow_authoring.md)
- [Agent 节点配置](../nodes/agent.md)

229
docs/user_guide/zh/execution_logic.md Executable file
View File

@ -0,0 +1,229 @@
# 图执行逻辑
> 版本：2025-12-16
本文档详细说明 DevAll 后端如何解析和执行工作流图，特别是对于包含循环结构的复杂图的处理机制。
## 1. 执行引擎概述
DevAll 工作流执行引擎支持两类图结构：
| 图类型 | 特征 | 执行策略 |
|--------|------|----------|
| **DAG（有向无环图）** | 节点间无循环依赖 | 拓扑排序 + 同层并发执行 |
| **含环有向图** | 存在一个或多个循环结构 | 递归式超级节点调度 |
执行引擎会自动检测图结构，选择合适的执行策略。
## 2. DAG 执行流程
对于不包含循环的工作流图，执行引擎采用标准的 DAG 调度策略：
1. **构建前驱/后继关系**：解析边定义，为每个节点建立 `predecessors` 和 `successors` 列表
2. **计算入度**：统计每个节点的前驱数量
3. **拓扑排序**：将入度为 0 的节点放入第一层，执行后将后继节点入度减 1，新的入度为 0 节点进入下一层
4. **同层并发**：同一层内的节点无依赖关系，可以并行执行
```mermaid
flowchart LR
subgraph Layer1["执行层 1"]
A["节点 A"]
B["节点 B"]
end
subgraph Layer2["执行层 2"]
C["节点 C"]
end
subgraph Layer3["执行层 3"]
D["节点 D"]
end
A --> C
B --> C
C --> D
```
## 3. 循环图执行流程
### 3.1 Tarjan 强连通分量检测
当图中存在循环结构时，执行引擎首先使用 **Tarjan 算法** 检测所有强连通分量（Strongly Connected Components, SCC）。Tarjan 算法通过深度优先搜索，在 O(|V|+|E|) 时间复杂度内识别图中的所有环路。
包含多于一个节点的 SCC 即为环路结构。
### 3.2 超级节点构建
检测到环路后，执行引擎将每个环路抽象为一个"超级节点"（Super Node）：
- 环路内部的所有节点被封装在超级节点中
- 超级节点之间的依赖关系来源于原始节点间的跨环边
- 封装后的超级节点图一定是 DAG，可以进行拓扑排序
```mermaid
flowchart TB
subgraph Original["原始图"]
direction TB
A1["A"] --> B1["B"]
B1 --> C1["C"]
C1 --> B1
C1 --> D1["D"]
end
subgraph Abstracted["超级节点图"]
direction TB
A2["节点 A"] --> S1["超级节点<br/>(B, C 环路)"]
S1 --> D2["节点 D"]
end
Original -.->|"抽象"| Abstracted
```
### 3.3 递归式环路执行策略
对于环路超级节点，系统采用递归式执行策略：
#### 步骤 1：唯一初始节点识别
分析环路边界，识别当前被唯一触发的入口节点作为"初始节点"。该节点必须满足：
- 被环路外部的前驱节点通过满足条件的边触发
- 有且仅有一个节点满足此条件
#### 步骤 2：构建作用域子图
以当前环路的所有节点为作用域，**逻辑上移除初始节点的所有入边**。这一操作打破外层环的边界，使后续的环路检测仅针对环内部的嵌套结构进行。
#### 步骤 3：嵌套环路检测
对构建的子图再次应用 Tarjan 算法，检测作用域内的嵌套环路。由于初始节点的入边已被移除，检测到的强连通分量仅为真正的内层嵌套环。
#### 步骤 4：内层超级节点构建与拓扑排序
若检测到嵌套环路：
- 将每个内层环路抽象为超级节点
- 构建作用域内的超级节点依赖图
- 对该超级节点图执行拓扑排序
若未检测到嵌套环路，则直接进行 DAG 拓扑排序。
#### 步骤 5：分层执行
按拓扑排序得到的执行层次依次执行：
- **普通节点**：检查触发状态后执行，首轮迭代时初始节点强制执行
- **内层环路超级节点**：**递归调用步骤 1-6**，形成嵌套执行结构
#### 步骤 6：退出条件检查
每完成一轮环内执行后，系统检查以下退出条件：
- **出口边触发**：若任一环内节点触发了环外节点的边，则退出环路
- **最大迭代次数**：若达到配置的最大迭代次数（默认 100），强制终止
- **初始节点未被重触发**：若初始节点未被环内前驱节点重新触发，环路自然终止
若条件均不满足，则返回步骤 2 开始下一轮迭代。
### 3.4 环路执行流程图
```mermaid
flowchart TB
A["环路超级节点被调度"] --> B["识别唯一触发的初始节点"]
B --> C{"是否有有效初始节点?"}
C -->|"无"| D["跳过该环路"]
C -->|"有多个"| E["报告配置错误"]
C -->|"唯一"| F["构建作用域子图<br/>移除初始节点入边"]
F --> G["Tarjan算法检测嵌套环路"]
G --> H{"存在内层嵌套环?"}
H -->|"否"| I["DAG拓扑排序"]
H -->|"是"| J["构建内层超级节点<br/>执行拓扑排序"]
I --> K["分层执行"]
J --> K
K --> L["执行普通节点"]
K --> M["递归执行内层环路"]
L --> N{"检查退出条件"}
M --> N
N -->|"出口边被触发"| O["退出环路"]
N -->|"达到最大迭代次数"| O
N -->|"初始节点未被重触发"| O
N -->|"继续迭代"| F
```
## 4. 边条件与触发机制
### 4.1 边触发（trigger）
每条边有一个 `trigger` 属性，决定该边是否参与执行顺序计算：
| trigger 值 | 行为 |
|------------|------|
| `true`（默认） | 该边参与拓扑排序，目标节点等待源节点完成 |
| `false` | 该边不参与拓扑排序，仅用于数据传递 |
### 4.2 边条件（condition）
边条件决定数据是否沿该边流动：
- `true`（默认）：总是传递
- `keyword`：检查上游输出是否包含/不包含特定关键词
- `function`：调用自定义函数判断
- 其他自定义条件类型
只有当条件满足时，目标节点才会被触发执行。
## 5. 典型循环场景示例
### 5.1 人工审阅循环
```yaml
nodes:
- id: Writer
type: agent
config:
name: gpt-4o
role: 你是一位专业的技术作家
- id: Reviewer
type: human
config:
description: 请审阅文章，满意请输入 ACCEPT
edges:
- from: Writer
to: Reviewer
- from: Reviewer
to: Writer
condition:
type: keyword
config:
none: [ACCEPT] # 不包含 ACCEPT 时继续循环
```
执行流程：
1. Writer 生成文章
2. Reviewer 人工审阅
3. 若输入不包含 "ACCEPT"，返回 Writer 修改
4. 若输入包含 "ACCEPT"，退出循环
### 5.2 嵌套循环
系统支持任意深度的嵌套循环。例如，一个外层"审阅-修订"循环内部可以包含一个"生成-验证"循环：
```
外层循环 (Writer -> Reviewer -> Writer)
└── 内层循环 (Generator -> Validator -> Generator)
```
递归式执行策略会自动处理这种嵌套结构。
## 6. 关键代码模块
| 模块 | 功能 |
|------|------|
| `workflow/cycle_manager.py` | Tarjan 算法实现、环路信息管理 |
| `workflow/topology_builder.py` | 超级节点图构建、拓扑排序 |
| `workflow/executor/cycle_executor.py` | 递归式环路执行器 |
| `workflow/graph.py` | 图执行主入口 |
## 7. 变更记录
- **2025-12-16**：新增图执行逻辑文档，详细说明 DAG 与循环图的执行策略。

61
docs/user_guide/zh/field_specs.md Executable file
View File

@ -0,0 +1,61 @@
# FIELD_SPECS 定义指南
本文档解释如何在新增配置（Config）时正确编写 `FIELD_SPECS`，以便 Web UI 表单和 `python -m tools.export_design_template` 命令自动生成可视化表单与 YAML 模板。本指南适用于所有继承 `BaseConfig` 的配置类，例如节点、Memory、Thinking、Tooling 等。
## 1. 为什么需要 FIELD_SPECS
- UI 表单依赖 `FIELD_SPECS` 生成输入控件、默认值、提示文案。
- 设计模板导出脚本会读取 `FIELD_SPECS`，将字段元数据写入 `yaml_template/design*.yaml` 以及 `frontend/public/` 镜像文件。
- 没有 `FIELD_SPECS` 的字段在前端无法展示，也不会出现在导出的模板中。
## 2. 基本结构
`FIELD_SPECS` 是一个 `{字段名: ConfigFieldSpec}` 的字典，通常定义在 Config 类内：
```python
FIELD_SPECS = {
"interpreter": ConfigFieldSpec(
name="interpreter",
display_name="解释器",
type_hint="str",
required=False,
default="python3",
description="Python 可执行文件路径",
),
...
}
```
核心字段说明：
- `name`：与 YAML 字段一致。
- `display_name`：可选的用户展示名称，前端表单优先显示；缺省时自动回退到 `name`。
- `type_hint`：供 UI/文档展示的类型描述，例如 `str`、`list[str]`、`dict[str, Any]`。
- `required`：是否必填；若有默认值通常设为 False。
- `default`：默认值（标量或 JSON 可序列化对象）。
- `description`：表单提示与文档说明。
- `enum`：可选值列表（字符串数组）。
- `enumOptions`：为枚举提供 label/description 等附加提示，推荐搭配 `enum` 一起返回，提升表单友好度。
- `child`：嵌套子配置类（引用另一个 `BaseConfig` 子类）。
## 3. 编写流程
1. **实现 `from_dict` 校验**：在解析 YAML 时确保类型正确、提供清晰错误（抛 `ConfigError`，见 `entity/configs/python_runner.py`）。
2. **定义 `FIELD_SPECS`**：覆盖所有公开字段，提供类型、描述、默认值等信息。
3. **动态字段处理**：若字段依赖注册表或目录扫描结果，重写 `field_specs()` 并使用 `replace()` 注入实时 `enum`/`description`（示例：`FunctionToolEntryConfig.field_specs()` 自动列出函数目录）。
4. **导出设计模板**：完成修改后运行：
```bash
python -m tools.export_design_template --output yaml_template/design.yaml --mirror frontend/public/design.yaml
```
该命令会根据最新的 `FIELD_SPECS` 生成 YAML 模板和前端镜像文件，无需手动编辑。
## 4. 常见模式示例
- **简单标量字段**：`entity/configs/python_runner.py` 中的 `timeout_seconds`，展示如何设置整数默认值和校验。
- **嵌套列表字段**：`entity/configs/memory.py` 的 `file_sources` 使用 `child=FileSourceConfig`，UI 自动渲染可重复子表单。
- **动态枚举**：`entity/configs/node.py:304` 的 `Node.field_specs()` 使用节点注册表填充 `type` 选项并附带 `enumOptions` 描述；`FunctionToolEntryConfig.field_specs()` 从函数目录生成带说明的枚举列表。
- **注册驱动描述**：调用 `register_node_type`/`register_memory_store`/`register_thinking_mode`/`register_tooling_type` 时提供的 `summary/description` 会自动写入 `enumOptions`，务必填写，避免前端看到没有含义的值。
- **可选区块**：通过 `required=False` + `default=...` 表示可选配置，`from_dict` 中需妥善处理。
## 5. 最佳实践
- 描述保持用户友好，明确单位（例如“超时时间（秒）”）。
- 默认值应与 `from_dict` 行为一致，避免 UI 默认与后端解析不符。
- 对嵌套配置提供精简示例或引用，以免 UI 难以理解字段含义。
- 修改或新增 `FIELD_SPECS` 后，记得同步导出设计模板。
如需更多例子，可查阅 `entity/configs/model.py`、`entity/configs/tooling.py` 等文件，或参考已有节点/Memory/Thinking 配置的实现。

45
docs/user_guide/zh/index.md Executable file
View File

@ -0,0 +1,45 @@
# DevAll 后端用户文档
本目录作为导航页，面向需要部署、编排或扩展 DevAll 后端的读者。详尽步骤与示例请在下表中找到目标子文档。
## 1. 文档地图
| 主题 | 内容提要 |
| --- |-----------------------------------------------------------|
| [Web UI 快速入门](web_ui_guide.md) | 前端界面操作、工作流执行、人工审阅、故障排查 |
| [工作流编排](workflow_authoring.md) | YAML 结构、节点类型、Provider/边条件、设计模板导出、CLI 运行 |
| [图执行逻辑](execution_logic.md) | DAG/循环图执行策略、Tarjan 环路检测、超级节点构建、递归式环路执行 |
| [Dynamic 并行执行](dynamic_execution.md) | Map/Tree 模式、Split 拆分策略、并行处理与层级归约 |
| [Memory 模块](modules/memory.md) | Memory 列表架构、内置 `simple`/`file`/`blackboard` 行为、嵌入配置、排障 |
| [Thinking 模块](modules/thinking.md) | 思考增强机制、自我反思模式、扩展自定义思考模式 |
| [Tooling 模块](modules/tooling/README.md) | Function / MCP 模式、上下文注入、内置函数清单、MCP 启动方式 |
| [节点类型详解](nodes/) | Agent、Python、Human、Subgraph、Passthrough、Literal、Loop Counter 等节点配置 |
| [附件与工件 API](attachments.md) | 上传/列举/下载接口、manifest 结构、清理策略、安全限制 |
| [FIELD_SPECS 规范](field_specs.md) | UI 表单与模板导出的字段元数据标准（如果您希望自定义模块，请务必阅读此文档） |
| [配置 Schema API 契约](config_schema_contract.md) | `/api/config/schema(*)` 请求示例、breadcrumbs 协议（用户可忽略） |
## 2. 产品概览（后端视角）
- **工作流调度引擎**：解析 YAML DAG，在统一上下文中协调 `model`、`python`、`tooling`、`human` 等节点，并把节点输出写入 `WareHouse/<session>/`。
- **多 Provider 抽象**：`runtime/node/agent/providers/` 层封装 OpenAI、Gemini 等 API，可在节点级别切换模型与鉴权，亦支持额外 `thinking` 与 `memories` 配置。
- **实时可观测性**：FastAPI + WebSocket 将节点状态、stdout/stderr、工件事件推送至 Web UI；结构化日志写入 `logs/`，便于集中收集。
- **运行资产管理**：每次运行创建独立 Session，附件、Python workspace、context snapshot、输出摘要等均可下载。
## 3. 架构与运行流概览
1. **入口**：Web UI 与 CLI 调用 `server_main.py` 暴露的 FastAPI（如 `/api/workflow/execute`）。
2. **验证/入队**：`WorkflowRunService` 校验 YAML、创建 Session、准备 `code_workspace/attachments/`，随后调度器在 `workflow/` 中运行 DAG。
3. **执行阶段**：节点执行器负责依赖解析、上下文传递、工具调用、memory 检索；`MemoryManager`、`ToolingConfig`、`ThinkingManager` 会在模型节点内按需触发。
4. **可观测性**：WebSocket 推送状态、日志、artifact 事件；`logs/` 存储 JSON 日志，`WareHouse/` 保存运行资产。
5. **清理与下载**：Session 结束后可选择打包下载或通过附件 API 逐项获取；保留策略由部署者自定。
## 4. 角色导航
- **解决方案工程师/Prompt 工程师**：从 [工作流编排](workflow_authoring.md) 入手，若需要上下文记忆或工具扩展，分别阅读 Memory 与 Tooling 模块文档。
- **扩展开发者**：结合 [FIELD_SPECS](field_specs.md) 与 [Tooling 模块](modules/tooling/README.md) 了解注册流程，必要时参照 [配置 Schema API 契约](config_schema_contract.md) 调试 UI 交互（英文版见 `docs/en/config_schema_contract.md`）。
## 5. 常用术语
- **Session**：一次完整运行的 ID（由时间戳+名称组成），贯穿 Web UI、后端和 `WareHouse/`。
- **code_workspace**：Python 节点共享的目录，位于 `WareHouse/<session>/code_workspace/`，包含自动同步的附件。
- **Attachment**：用户上传或运行期间注册的文件，通过 REST/WS API 可查询/下载。
- **Memory Store / Attachment**：Memory Store 定义存储实现；Memory Attachment 是模型节点引用 Memory Store 的规则（检索阶段、读写策略等）。
- **Tooling**：模型节点绑定的工具执行环境（Function 或 MCP）。
如发现内容缺失或过时，请在仓库提交 Issue/PR，或在 docs 目录内直接补充并同步至前端模板。

122
docs/user_guide/zh/modules/memory.md Executable file
View File

@ -0,0 +1,122 @@
# Memory 模块指南
本文档解释 DevAll 的 Memory 体系：memory 列表配置、内置存储实现、Agent 节点如何引用记忆，以及排障建议。代码主要位于 `entity/configs/memory.py`、`node/agent/memory/*.py`。
## 1. 体系结构
1. **Memory Store**：在 YAML `memory[]` 中声明，包含 `name`、`type` 和 `config`。`type` 由 `register_memory_store()` 注册，并映射到具体实现。
2. **Memory Attachment**：在 Agent 节点（`AgentConfig.memories`）中引用 `MemoryAttachmentConfig`，指定读取/写入策略及检索阶段。
3. **MemoryManager**：运行期根据 Attachment+Store 构建 Memory 实例，负责 `load()`、`retrieve()`、`update()`、`save()`。
4. **Embedding**：`SimpleMemoryConfig`、`FileMemoryConfig` 可内嵌 `EmbeddingConfig`，由 `EmbeddingFactory` 创建 OpenAI 或本地向量模型。
## 2. Memory 配置示例
```yaml
memory:
- name: convo_cache
type: simple
config:
memory_path: WareHouse/shared/simple.json
embedding:
provider: openai
model: text-embedding-3-small
api_key: ${API_KEY}
- name: project_docs
type: file
config:
index_path: WareHouse/index/project_docs.json
file_sources:
- path: docs/
file_types: [".md", ".mdx"]
recursive: true
embedding:
provider: openai
model: text-embedding-3-small
```
## 3. 内置 Memory Store 对比
| 类型 | 路径 | 特点 | 适用场景 |
| --- | --- | --- | --- |
| `simple` | `node/agent/memory/simple_memory.py` | 运行结束后可选择落盘（JSON）；使用向量搜索（FAISS）+语义重打分；支持读写 | 小规模对话记忆、快速原型 |
| `file` | `node/agent/memory/file_memory.py` | 将指定文件/目录切片为向量索引，只读；自动检测文件变更并更新索引 | 知识库、文档问答 |
| `blackboard` | `node/agent/memory/blackboard_memory.py` | 轻量附加日志，按时间/条数裁剪；不依赖向量检索 | 简易广播板、流水线调试 |
> 所有内置 store 都会在 `register_memory_store()` 中注册，摘要可通过 `MemoryStoreConfig.field_specs()` 在 UI 中展示。
## 4. MemoryAttachmentConfig 说明
| 字段 | 说明 |
| --- | --- |
| `name` | 引用的 Memory Store 名称（需在 `stores[]` 中存在且唯一）。|
| `retrieve_stage` | 可选数组，限制检索发生的阶段（`AgentExecFlowStage`：`pre`, `plan`, `gen`, `critique` 等）。缺省表示所有阶段。|
| `top_k` | 每次检索返回的条数，默认 3。|
| `similarity_threshold` | 过滤相似度下限（-1 表示不限制）。|
| `read` / `write` | 是否允许在该节点读取/写回此记忆。|
Agent 节点示例：
```yaml
nodes:
- id: answer
type: agent
config:
provider: openai
model: gpt-4o-mini
prompt_template: answer_user
memories:
- name: convo_cache
retrieve_stage: ["gen"]
top_k: 5
read: true
write: true
- name: project_docs
read: true
write: false
```
执行顺序：
1. `MemoryManager` 在节点进入 `gen` 阶段时，遍历 Attachments。
2. 满足阶段与 `read=true` 的 Attachment 调用对应 Memory Store 的 `retrieve()`。
3. 结果格式化并拼接为“===== 相关记忆 =====”文本写入 Agent 输入上下文。
4. 节点完成后，`write=true` 的 Attachment 将调用 `update()` 并在必要时 `save()`。
## 5. Store 细节
所有 Memory Store 都持久化统一的 `MemoryItem` 结构：
- `content_summary`：用于检索的精简文本；
- `input_snapshot` / `output_snapshot`：序列化的消息块（含 base64 附件），确保多模态上下文不会丢失；
- `metadata`：记录角色、输入预览、附件 ID 等附加信息。
这使得 Memory 与 Thinking 模块可以共享多模态内容，无需额外适配。
### 5.1 SimpleMemory
- **路径**：`SimpleMemoryConfig.memory_path`（可为 `auto`），缺省仅驻留内存。
- **检索**：
1. 以 prompt 构建查询文本并做裁剪。
2. 调用 Embedding 生成向量 → FAISS `IndexFlatIP` 检索 → 语义重打分（Jaccard/LCS）。
- **写入**：`update()` 根据输入/输出生成 `MemoryContentSnapshot`，计算摘要哈希去重，再写入 embedding + snapshot + 附件元信息。
- **适配建议**：控制 `max_content_length` 避免爆 context；结合 `top_k`/`similarity_threshold` 防止无关内容。
### 5.2 FileMemory
- **配置**：至少一个 `file_sources`（路径、后缀过滤、递归、编码）。`index_path` 必填，方便增量更新。
- **索引流程**：扫描文件 → 切片（默认 500 字符、重叠 50）→ Embedding → 写入 JSON（包括 `file_metadata`）。
- **检索**：同样使用 FAISS 余弦相似度，只读，不支持 `update()`。
- **维护**：`load()` 时校验文件哈希，必要时重建索引；建议将 `index_path` 放在持久卷。
### 5.3 BlackboardMemory
- **配置**：`memory_path`（可 `auto`）、`max_items`。若路径不存在则在 Session 目录内创建。
- **检索**：直接返回最近 `top_k` 条，按时间排序。
- **写入**：`update()` 以 append 方式存储最新的输入/输出 snapshot（文本 + 块 + 附件信息），不生成向量，适合事件流或人工批注。
## 6. EmbeddingConfig 提示
- 字段：`provider`, `model`, `api_key`, `base_url`, `params`。
- `provider=openai` 时使用 `openai.OpenAI` 客户端，可配置 `base_url` 以兼容兼容层。
- `params` 支持 `use_chunking`, `chunk_strategy`, `max_length` 等自定义键。
- `provider=local` 时需提供 `params.model_path`，依赖 `sentence-transformers`。
## 7. 排错与最佳实践
- **重复命名**：内存列表会校验 `memory[]` 名称唯一；重复时抛出 `ConfigError`。
- **缺少 embedding**：`SimpleMemory`/`FileMemory` 若未提供 embedding，则仅能以追加方式工作（SimpleMemory）或抛出错误（FileMemory）。
- **权限**：确保 `memory_path`/`index_path` 所在目录可写；容器化部署应挂载卷。
- **性能**：
- 大型 FileMemory 建议离线构建索引并缓存。
- 通过 `retrieve_stage` 控制检索次数，减少模型输入冗余。
- 调整 `top_k`、`similarity_threshold` 以平衡召回与 token 成本。
## 8. 扩展自定义 Memory
1. 新建 Config + Store（继承 `MemoryBase`）。
2. 在 `node/agent/memory/registry.py` 中调用 `register_memory_store("my_store", config_cls=..., factory=..., summary="用途")`。
3. 补充 `FIELD_SPECS`，运行 `python -m tools.export_design_template ...` 以让前端获取新枚举。
4. 更新本指南或附带 README，说明新 store 的配置项与边界条件。

108
docs/user_guide/zh/modules/thinking.md Executable file
View File

@ -0,0 +1,108 @@
# Thinking 模块指南
Thinking 模块为 Agent 节点提供思考增强能力，使模型能够在生成结果前或后进行额外的推理过程。本文档介绍 Thinking 模块的架构、内置模式及配置方法。
## 1. 体系结构
1. **ThinkingConfig**：在 YAML `nodes[].config.thinking` 中声明，包含 `type` 和 `config` 两个字段。
2. **ThinkingManagerBase**：抽象基类，定义 `_before_gen_think` 和 `_after_gen_think` 两个时机的思考逻辑。
3. **注册中心**：通过 `register_thinking_mode()` 注册新的思考模式，Schema API 会自动展示可用选项。
## 2. 配置示例
```yaml
nodes:
- id: Thoughtful Agent
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
thinking:
type: reflection
config:
reflection_prompt: |
请仔细审视你的回答，考虑以下方面：
1. 逻辑是否严密
2. 有无事实错误
3. 表达是否清晰
然后给出改进后的回答。
```
## 3. 内置思考模式
| 类型 | 描述 | 触发时机 | 配置字段 |
|------|------|----------|----------|
| `reflection` | 模型生成后进行自我反思并优化输出 | 生成后 (`after_gen`) | `reflection_prompt` |
### 3.1 Reflection 模式
Self-Reflection 模式让模型在初次生成后对自己的输出进行反思和改进。实现流程：
1. Agent 节点正常调用模型生成初始回答
2. ThinkingManager 将对话历史（系统角色、用户输入、模型输出）拼接为反思上下文
3. 结合 `reflection_prompt` 再次调用模型生成反思结果
4. 反思结果替换原始输出作为节点最终输出
#### 配置项
| 字段 | 类型 | 必填 | 说明 |
|------|------|------|------|
| `reflection_prompt` | string | 是 | 引导模型反思的提示词，可指定反思维度和期望改进方向 |
#### 适用场景
- **写作润色**：让模型自我审阅并修正语法、逻辑问题
- **代码审查**：生成代码后自动进行安全和质量检查
- **复杂推理**：对多步骤推理结果进行验证和修正
## 4. 执行时机
ThinkingManager 支持两种执行时机：
| 时机 | 属性 | 说明 |
|------|------|------|
| 生成前 (`before_gen`) | `before_gen_think_enabled` | 在模型调用前执行思考，可预处理输入 |
| 生成后 (`after_gen`) | `after_gen_think_enabled` | 在模型输出后执行思考，可后处理或优化输出 |
内置的 `reflection` 模式仅启用生成后思考。扩展开发者可根据需求实现生成前思考。
## 5. 与 Memory 的交互
Thinking 模块可访问 Memory 上下文：
- `ThinkingPayload.text`：当前阶段的文本内容
- `ThinkingPayload.blocks`：多模态内容块（图片、附件等）
- `ThinkingPayload.metadata`：附加元数据
Memory 检索结果会通过 `memory` 参数传入思考函数，允许反思时参考历史记忆。
## 6. 扩展自定义思考模式
1. **创建配置类**：继承 `BaseConfig`，定义所需配置字段
2. **实现 ThinkingManager**：继承 `ThinkingManagerBase`，实现 `_before_gen_think` 或 `_after_gen_think`
3. **注册模式**：
```python
from runtime.node.agent.thinking.registry import register_thinking_mode
register_thinking_mode(
"my_thinking",
config_cls=MyThinkingConfig,
manager_cls=MyThinkingManager,
summary="自定义思考模式描述",
)
```
4. **导出模板**：运行 `python -m tools.export_design_template` 更新前端选项
## 7. 最佳实践
- **控制反思轮次**：当前反思为单轮，若需多轮可在 `reflection_prompt` 中明确迭代要求
- **简洁提示词**：过长的 `reflection_prompt` 会增加 token 消耗，建议聚焦关键改进点
- **配合 Memory**：将重要反思结果存入 Memory，供后续节点参考
- **监控成本**：反思会额外调用模型，注意 token 用量
## 8. 相关文档
- [Agent 节点配置](../nodes/agent.md)
- [Memory 模块](memory.md)
- [工作流编排指南](../workflow_authoring.md)

47
docs/user_guide/zh/modules/tooling/README.md Executable file
View File

@ -0,0 +1,47 @@
# Tooling 模块总览
DevAll 目前支持两类工具绑定到 Agent 节点：
1. **Function Tooling**：调用仓库内的 Python 函数（`functions/function_calling/`），通过 JSON Schema 自动生成工具签名。
2. **MCP Tooling**：连接符合 Model Context Protocol 的外部服务，可直接复用 FastMCP、Claude Desktop 等工具生态。
所有 Tooling 配置都挂载在 `AgentConfig.tooling`：
```yaml
nodes:
- id: solve
type: agent
config:
provider: openai
model: gpt-4o-mini
prompt_template: solver
tooling:
type: function
config:
tools:
- name: describe_available_files
- name: load_file
auto_load: true
timeout: 20
```
## 1. 生命周期
1. 解析阶段：`ToolingConfig` 根据 `type` 选择 `FunctionToolConfig`、`McpRemoteConfig` 或 `McpLocalConfig`，字段定义来自 `entity/configs/tooling.py`。
2. 运行阶段：Agent 节点根据响应启用工具调用；当 LLM 选择某工具时，执行器会将 `_context`（附件仓库、workspace 路径等）注入函数或通过 MCP 发送请求。
3. 结束阶段：工具输出写入 Agent 消息流，必要时注册为附件（如 `load_file`）。
## 2. 文档结构
- [function.md](function.md)：Function Tooling 配置、上下文注入、最佳实践。
- [function_catalog.md](function_catalog.md)：仓库内置函数清单与示例。
- [mcp.md](mcp.md)：MCP 工具配置、自动启动、FastMCP 示例、安全提示。
## 3. 快速对比
| 维度 | Function | MCP |
| --- | --- | --- |
| 部署 | 同进程调用本地 Python 函数 | Remote：直连 HTTP 服务；Local：拉起本地进程并通过 stdio 连接 |
| Schemas | 自动从类型注解 + `ParamMeta` 生成 | 由 MCP JSON Schema 提供 |
| 上下文 | 自动注入 `_context`（附件/workspace） | 取决于 MCP 服务器实现 |
| 典型用途 | 文件操作、本地脚本、内部 API | 第三方工具合集、浏览器、数据库代理 |
## 4. 安全提示
- Function Tooling 运行在后端进程中，应确保函数遵循最小权限原则；不要在函数中执行不受控的命令。
- MCP Tooling 分为 **Remote (HTTP)** 与 **Local (stdio)**。Remote 仅配置已有服务器地址；Local 会拉起进程，请使用受控脚本并限制环境变量，必要时通过 `wait_for_log` 等字段判断进程是否就绪。
- 若工具可能修改附件或 workspace，请结合 [附件指南](../../attachments.md) 了解生命周期与清理策略。

80
docs/user_guide/zh/modules/tooling/function.md Executable file
View File

@ -0,0 +1,80 @@
# Function Tooling 配置指南
`FunctionToolConfig` 允许 Agent 节点调用仓库中的 Python 函数。相关代码位于 `entity/configs/tooling.py`、`utils/function_catalog.py` 以及 `functions/function_calling/`。
## 1. 配置字段
| 字段 | 说明 |
| --- | --- |
| `tools` | 列表，元素为 `FunctionToolEntryConfig`。每个条目至少包含 `name`。|
| `timeout` | 单次工具执行的超时时间（秒）。|
`FunctionToolEntryConfig` 字段：
- `name`：函数名，来自 `functions/function_calling/` 文件的顶级函数。
### 函数列表展示与 `module_name:All`
- UI 下拉列表会将每个函数展示为 `module_name:function_name`。`module_name` 等于函数文件相对于 `functions/function_calling/` 的路径（去掉 `.py`，子目录使用 `/` 连接），便于快速定位语义相关模块。
- 每个模块顶部都会自动插入 `module_name:All` 选项，并且所有模块的 `All` 条目按照字典序排在列表最前。选择该项时会在解析阶段展开为该模块下的所有函数，顺序同样遵循字典序。
- `module_name:All` 只能批量引入函数，禁止同时填写 `description`、`parameters` 或 `auto_fill` 等覆盖字段；若需要自定义，请展开后针对具体函数单独配置。
- 函数与模块都采用全局字典序排列，使长列表更易检索；YAML 中仍然以真实函数名落盘，`module_name:All` 仅作为输入辅助。
## 2. 函数目录要求
- 路径：`functions/function_calling/`（可通过 `MAC_FUNCTIONS_DIR` 覆盖）。
- 每个函数：
- 必须位于模块顶层。
- 使用 Python 类型注解；若需枚举或描述，可使用 `typing.Annotated[..., ParamMeta(...)]`。
- 不允许以 `_` 开头的参数暴露给 Agent；`*_args`、`**kwargs` 会被过滤。
- 可以通过 docstring 的首段提供描述（自动截断为 600 字符）。
- `utils/function_catalog.py` 会在启动时生成 JSON Schema，并向前端/CLI 暴露。
## 3. 上下文注入
执行器会对被调用的函数提供 `_context` 关键字参数，包含：
| 键 | 值 |
| --- | --- |
| `attachment_store` | `utils.attachments.AttachmentStore` 实例，可查询/注册附件。|
| `python_workspace_root` | 当前 Session 的 `code_workspace/`。|
| `graph_directory` | Session 根目录，可推导相对路径。|
| `human_prompt` | `utils.human_prompt.HumanPromptService`，可调用 `request()` 触发人工反馈。|
| 其他 | 视运行环境扩展，例如 `session_id`、`node_id`。|
函数可声明 `_context: dict | None = None` 并自行解析（参考 `functions/function_calling/file.py` 中的 `FileToolContext`，还可参考 `functions/function_calling/user.py`）。
## 4. 示例：文件读取工具
```python
from typing import Annotated
from utils.function_catalog import ParamMeta
def read_text_file(
path: Annotated[str, ParamMeta(description="workspace 相对路径")],
*,
encoding: str = "utf-8",
_context: dict | None = None,
) -> str:
ctx = FileToolContext(_context)
target = ctx.resolve_under_workspace(path)
return target.read_text(encoding=encoding)
```
在 YAML 中引用：
```yaml
nodes:
- id: summarize
type: agent
config:
tooling:
type: function
config:
tools:
- name: describe_available_files
- name: read_text_file
```
## 5. 扩展流程
1. 在 `functions/function_calling/` 新建模块或函数。
2. 使用类型注解 + `ParamMeta` 描述参数；如需禁止自动 Schema，可设置 `auto_fill: false` 并提供手写 `parameters`。
3. 若函数依赖额外第三方库，可在仓库 `requirements.txt`/`pyproject.toml` 中声明，或在函数内调用 `install_python_packages`（同目录提供）动态安装。
4. 运行 `python -m tools.export_design_template ...` 以刷新前端枚举。
## 6. 调试与排错
- 若前端/CLI 报告 “function 'xxx' not found”，检查函数名称与文件是否位于 `MAC_FUNCTIONS_DIR`（默认 `functions/function_calling/`）。
- `function_catalog` 加载失败时，`FunctionToolEntryConfig.field_specs()` 会在描述中提示错误，请先修复函数语法或依赖。
- 工具运行超时会向 Agent 返回异常文本；可通过 `timeout` 扩大限额，或在函数内部自行捕获并返回友好错误。

168
docs/user_guide/zh/modules/tooling/function_catalog.md Executable file
View File

@ -0,0 +1,168 @@
# 内置 Function 工具目录
本文档列出 `functions/function_calling/` 目录中预置的所有工具，供 Agent 节点通过 Function Tooling 调用。
## 快速导入
在 YAML 中可通过以下方式引用：
```yaml
tooling:
- type: function
config:
tools:
- name: file:All # 导入整个模块
- name: save_file # 导入单个函数
- name: deep_research:All
```
---
## 文件操作 (file.py)
文件与目录操作工具集，用于在 `code_workspace/` 中进行文件管理。
| 函数 | 说明 |
|------|------|
| `describe_available_files` | 列出附件仓库和 code_workspace 中的可用文件 |
| `list_directory` | 列出指定目录内容 |
| `create_folder` | 创建文件夹（支持多级目录） |
| `delete_path` | 删除文件或目录 |
| `load_file` | 加载文件并注册为附件，支持多模态（文本/图片/音频） |
| `save_file` | 保存文本内容到文件 |
| `read_text_file_snippet` | 读取文本片段（offset + limit），适合大文件 |
| `read_file_segment` | 按行范围读取文件，支持行号元数据 |
| `apply_text_edits` | 应用多处文本编辑，保留换行符和编码 |
| `rename_path` | 重命名文件或目录 |
| `copy_path` | 复制文件或目录树 |
| `move_path` | 移动文件或目录 |
| `search_in_files` | 在工作区文件中搜索文本或正则模式 |
**示例 YAML**：[ChatDev_v1.yaml](../../../../../yaml_instance/ChatDev_v1.yaml)、[file_tool_use_case.yaml](../../../../../yaml_instance/file_tool_use_case.yaml)
---
## Python 环境管理 (uv_related.py)
使用 uv 管理 Python 环境和依赖。
| 函数 | 说明 |
|------|------|
| `install_python_packages` | 使用 `uv add` 安装 Python 包 |
| `init_python_env` | 初始化 Python 环境（uv lock + venv） |
| `uv_run` | 在工作区内执行 uv run，运行模块或脚本 |
**示例 YAML**：[ChatDev_v1.yaml](../../../../../yaml_instance/ChatDev_v1.yaml)
---
## 深度研究 (deep_research.py)
搜索结果管理与报告生成工具，适用于自动化研究场景。
### 搜索结果管理
| 函数 | 说明 |
|------|------|
| `search_save_result` | 保存或更新搜索结果（URL、标题、摘要、详情） |
| `search_load_all` | 加载所有已保存的搜索结果 |
| `search_load_by_url` | 按 URL 加载特定搜索结果 |
| `search_high_light_key` | 为搜索结果保存高亮关键词 |
### 报告管理
| 函数 | 说明 |
|------|------|
| `report_read` | 读取报告完整内容 |
| `report_read_chapter` | 读取特定章节（支持多级路径如 `Intro/Background`） |
| `report_outline` | 获取报告大纲（标题层级结构） |
| `report_create_chapter` | 创建新章节 |
| `report_rewrite_chapter` | 重写章节内容 |
| `report_continue_chapter` | 追加内容到现有章节 |
| `report_reorder_chapters` | 重新排序章节 |
| `report_del_chapter` | 删除章节 |
| `report_export_pdf` | 导出报告为 PDF |
**示例 YAML**：[deep_research_v1.yaml](../../../../../yaml_instance/deep_research_v1.yaml)
---
## 网络工具 (web.py)
网络搜索与网页内容获取。
| 函数 | 说明 |
|------|------|
| `web_search` | 使用 Serper.dev 执行网络搜索，支持分页和多语言 |
| `read_webpage_content` | 使用 Jina Reader 读取网页内容，支持速率限制 |
**环境变量**：
- `SERPER_DEV_API_KEY`：Serper.dev API 密钥
- `JINA_API_KEY`：Jina API 密钥（可选，无密钥时自动限速 20 RPM）
**示例 YAML**：[deep_research_v1.yaml](../../../../../yaml_instance/deep_research_v1.yaml)
---
## 视频工具 (video.py)
Manim 动画渲染与视频处理。
| 函数 | 说明 |
|------|------|
| `render_manim` | 渲染 Manim 脚本，自动检测场景类并输出视频 |
| `concat_videos` | 使用 FFmpeg 拼接多个视频文件 |
**示例 YAML**：[teach_video.yaml](../../../../../yaml_instance/teach_video.yaml)、[teach_video.yaml](../../../../../yaml_instance/teach_video.yaml)
---
## 代码执行 (code_executor.py)
| 函数 | 说明 |
|------|------|
| `execute_code` | 执行 Python 代码字符串，返回 stdout 和 stderr |
> ⚠️ **安全提示**：此工具具有高权限，应仅在可信工作流内使用。
---
## 用户交互 (user.py)
| 函数 | 说明 |
|------|------|
| `call_user` | 向用户发送指令并获取响应，用于需要人工输入的场景 |
---
## 天气查询 (weather.py)
示例工具，用于演示 Function Calling 流程。
| 函数 | 说明 |
|------|------|
| `get_city_num` | 返回城市编号（硬编码示例） |
| `get_weather` | 根据城市编号返回天气信息（硬编码示例） |
---
## 添加自定义工具
1. 在 `functions/function_calling/` 目录下创建 Python 文件
2. 使用类型注解定义参数：
```python
from typing import Annotated
from utils.function_catalog import ParamMeta
def my_tool(
param1: Annotated[str, ParamMeta(description="参数描述")],
*,
_context: dict | None = None, # 可选，系统自动注入
) -> str:
"""函数描述（会显示给 LLM）"""
return "result"
```
3. 重启后端服务器
4. 在 Agent 节点中通过 `name: my_tool` 或 `name: my_module:All` 引用

92
docs/user_guide/zh/modules/tooling/mcp.md Executable file
View File

@ -0,0 +1,92 @@
# MCP Tooling 指南
MCP 工具被明确拆分为 **Remote (HTTP)** 与 **Local (stdio)** 两种模式，对应 `tooling.type: mcp_remote` 与 `tooling.type: mcp_local`。旧的 `type: mcp` schema 已下线，请在 YAML 与文档中全部迁移。
## 1. 配置模式概览
| 模式 | Tooling type | 适用场景 | 关键字段 |
| --- | --- | --- | --- |
| Remote | `mcp_remote` | 已部署的 HTTP(S) MCP 服务器（如 FastMCP、Claude Desktop Connector、自建代理） | `server`、`headers`、`timeout` |
| Local | `mcp_local` | 通过 stdio 握手的本地可执行脚本（Blender MCP、CLI 工具等） | `command`、`args`、`cwd`、`env` 等进程字段 |
## 2. `McpRemoteConfig` 字段
| 字段 | 说明 |
| --- | --- |
| `server` | 必填，MCP HTTP(S) 端点，例如 `https://api.example.com/mcp`。 |
| `headers` | 可选，附加 HTTP 头（如 `Authorization`）。 |
| `timeout` | 可选，单次工具调用超时时间（秒）。 |
**YAML 示例：**
```yaml
nodes:
- id: remote_mcp
type: agent
config:
tooling:
type: mcp_remote
config:
server: https://mcp.mycompany.com/mcp
headers:
Authorization: Bearer ${MY_MCP_TOKEN}
timeout: 15
```
DevAll 会在列举/调用工具时连接该 URL，并携带 `headers`。若服务器不可达，将直接抛出错误，不再尝试本地回退。
## 3. `McpLocalConfig` 字段
`mcp_local` 直接在 `config` 下声明进程参数：
- `command` / `args`：可执行文件与参数（如 `uvx blender-mcp`）。
- `cwd`：可选工作目录。
- `env` / `inherit_env`：定制子进程环境；默认继承父进程后再覆盖。
- `startup_timeout`：等待 `wait_for_log` 命中的最长秒数。
- `wait_for_log`：stdout 正则，用于判定“就绪”。
**YAML 示例：**
```yaml
nodes:
- id: local_mcp
type: agent
config:
tooling:
type: mcp_local
config:
command: uvx
args:
- blender-mcp
cwd: ${REPO_ROOT}
wait_for_log: "MCP ready"
startup_timeout: 8
```
运行期间 DevAll 会保持该进程常驻，并通过 stdio 传输 MCP 数据帧。
## 4. FastMCP 示例服务器
`mcp_example/mcp_server.py`：
```python
from fastmcp import FastMCP
import random
mcp = FastMCP("Company Simple MCP Server", debug=True)
@mcp.tool
def rand_num(a: int, b: int) -> int:
return random.randint(a, b)
if __name__ == "__main__":
mcp.run()
```
启动：
```bash
uv run fastmcp run mcp_example/mcp_server.py --transport streamable-http --port 8010
```
- 若以 Remote 模式使用，只需将 `server` 指向 `http://127.0.0.1:8010/mcp`。
- 若以 Local 模式使用，可将 `command` 设置为 `uv run fastmcp run ...` 并保持 `transport=stdio`。
## 5. 安全与运维
- **网络暴露**：Remote 模式建议置于 HTTPS 反向代理之后，并结合 API Key/ACL；Local 模式进程仍可访问宿主机文件，请限制其权限。
- **资源回收**：Local 模式由 DevAll 负责终止子进程，确保脚本可以正确处理 SIGTERM/SIGKILL。
- **日志定位**：为 `wait_for_log` 输出清晰的“ready”日志，便于在超时时排查。
- **鉴权**：Remote 模式通过 `headers` 传递 Token；Local 模式可在 `env` 中注入密钥，注意不要写入仓库。
- **多会话**：MCP 服务若不支持多客户端，可在模型或工具层设置 `max_concurrency=1` 并在 YAML 中复用同一配置。
## 6. 调试步骤
1. Remote：使用 curl 或 `fastmcp client` 测试 HTTP 端点；Local：先单独运行并确认 stdout 中有 `wait_for_log` 匹配的文本。
2. 启动 DevAll（可加 `--reload`），观察后端日志是否打印工具清单。
3. 若调用失败，查看 Web UI 中的工具请求/响应，或在 `logs/` 中搜索对应 session 的结构化日志。

152
docs/user_guide/zh/nodes/agent.md Executable file
View File

@ -0,0 +1,152 @@
# Agent 节点
Agent 节点是 DevAll 平台中最核心的节点类型，用于调用大语言模型 (LLM) 完成文本生成、对话、推理等任务。它支持多种模型提供商（OpenAI、Gemini 等），并可配置工具调用、思维链、记忆等高级功能。
## 配置项
| 字段 | 类型 | 必填 | 默认值 | 说明 |
|------|------|------|--------|------|
| `provider` | string | 是 | `openai` | 模型提供商名称，如 `openai`、`gemini` |
| `name` | string | 是 | - | 模型名称，如 `gpt-4o`、`gemini-2.0-flash-001` |
| `role` | text | 否 | - | 系统提示词 (System Prompt) |
| `base_url` | string | 否 | 提供商默认 | API 端点 URL，支持 `${VAR}` 占位符 |
| `api_key` | string | 否 | - | API 密钥，建议使用环境变量 `${API_KEY}` |
| `params` | dict | 否 | `{}` | 模型调用参数（temperature、top_p 等） |
| `tooling` | object | 否 | - | 工具调用配置，详见 [Tooling 模块](../modules/tooling/README.md) |
| `thinking` | object | 否 | - | 思维链配置，如 chain-of-thought、reflection |
| `memories` | list | 否 | `[]` | 记忆绑定配置，详见 [Memory 模块](../modules/memory.md) |
| `retry` | object | 否 | - | 自动重试策略配置 |
### 重试策略配置 (retry)
| 字段 | 类型 | 默认值 | 说明 |
|------|------|--------|------|
| `enabled` | bool | `true` | 是否启用自动重试 |
| `max_attempts` | int | `5` | 最大尝试次数（含首次） |
| `min_wait_seconds` | float | `1.0` | 最小退避等待时间 |
| `max_wait_seconds` | float | `6.0` | 最大退避等待时间 |
| `retry_on_status_codes` | list[int] | `[408,409,425,429,500,502,503,504]` | 触发重试的 HTTP 状态码 |
## 何时使用
- **文本生成**：写作、翻译、摘要、问答等
- **智能对话**：多轮对话、客服机器人
- **工具调用**：让模型调用外部 API 或执行函数
- **复杂推理**：配合 thinking 配置进行深度思考
- **知识检索**：配合 memories 实现 RAG 模式
## 示例
### 基础配置
```yaml
nodes:
- id: Writer
type: agent
config:
provider: openai
base_url: ${BASE_URL}
api_key: ${API_KEY}
name: gpt-4o
role: |
你是一位专业的技术文档撰写者，请用清晰简洁的语言回答问题。
params:
temperature: 0.7
max_tokens: 2000
```
### 配置工具调用
```yaml
nodes:
- id: Assistant
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
tooling:
type: function # 工具类型：function, mcp_remote, mcp_local
config:
tools: # 函数工具列表，来自 functions/function_calling/ 目录
- name: describe_available_files
- name: load_file
timeout: 20 # 可选：执行超时（秒）
```
### 配置 MCP 工具（Remote HTTP）
```yaml
nodes:
- id: MCP Agent
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
tooling:
type: mcp_remote
config:
server: http://localhost:8080/mcp # MCP 服务器端点
headers: # 可选：自定义请求头
Authorization: Bearer ${MCP_TOKEN}
timeout: 30 # 可选：请求超时（秒）
```
### 配置 MCP 工具（Local stdio）
```yaml
nodes:
- id: Local MCP Agent
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
tooling:
type: mcp_local
config:
command: uvx # 启动命令
args: ["mcp-server-sqlite", "--db-path", "data.db"]
cwd: ${WORKSPACE} # 可选，一般不需要配置
env: # 可选，一般不需要配置
DEBUG: "true"
startup_timeout: 10 # 可选：启动超时（秒）
```
### Gemini 多模态配置
```yaml
nodes:
- id: Vision Agent
type: agent
config:
provider: gemini
base_url: https://generativelanguage.googleapis.com
api_key: ${GEMINI_API_KEY}
name: gemini-2.5-flash-image
role: 你需要根据用户的输入，生成相应的图像内容。
```
### 配置重试策略
```yaml
nodes:
- id: Robust Agent
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
retry: # retry 默认启用，可以自己配置
enabled: true
max_attempts: 3
min_wait_seconds: 2.0
max_wait_seconds: 10.0
```
## 相关文档
- [Tooling 模块配置](../modules/tooling/README.md)
- [Memory 模块配置](../modules/memory.md)
- [工作流编排指南](../workflow_authoring.md)

114
docs/user_guide/zh/nodes/human.md Executable file
View File

@ -0,0 +1,114 @@
# Human 节点
Human 节点用于在工作流执行过程中引入人工交互，允许用户在 Web UI 中查看当前状态并提供输入。这种节点会阻塞工作流执行，直到用户提交响应。
## 配置项
| 字段 | 类型 | 必填 | 默认值 | 说明 |
|------|------|------|--------|------|
| `description` | text | 否 | - | 显示给用户的任务描述，说明需要人工完成的操作 |
## 核心概念
### 阻塞等待机制
当工作流执行到 Human 节点时：
1. 工作流暂停，等待人工输入
2. Web UI 显示当前上下文和任务描述
3. 用户在界面中输入响应
4. 工作流继续执行，将用户输入传递给下游节点
### 与 Web UI 交互
- Human 节点在 Launch 界面以对话形式呈现
- 用户可以查看之前的执行历史
- 支持附件上传（如文件、图片）
## 何时使用
- **审核确认**：让人工审核 LLM 输出后继续
- **修改意见**：收集用户对生成内容的修改建议
- **关键决策**：需要人工判断才能继续的分支
- **数据补充**：需要用户提供额外信息
- **质量把关**：在关键节点引入人工质检
## 示例
### 基础配置
```yaml
nodes:
- id: Human Reviewer
type: human
config:
description: 请审阅上述内容，如满意请输入 ACCEPT，否则输入修改意见。
```
### 人机协作循环
```yaml
nodes:
- id: Article Writer
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
role: 你是一位专业作家，根据用户要求撰写文章。
- id: Human Reviewer
type: human
config:
description: |
请审阅文章：
- 满意结果请输入 ACCEPT 结束流程
- 否则输入修改意见继续迭代
edges:
- from: Article Writer
to: Human Reviewer
- from: Human Reviewer
to: Article Writer
condition:
type: keyword
config:
none: [ACCEPT]
case_sensitive: false
```
### 多阶段审核
```yaml
nodes:
- id: Draft Generator
type: agent
config:
provider: openai
name: gpt-4o
- id: Content Review
type: human
config:
description: 请审核内容准确性，输入 APPROVED 或修改意见。
- id: Final Reviewer
type: human
config:
description: 最终确认，输入 PUBLISH 发布或 REJECT 驳回。
edges:
- from: Draft Generator
to: Content Review
- from: Content Review
to: Final Reviewer
condition:
type: keyword
config:
any: [APPROVED]
```
## 最佳实践
- 在 `description` 中清晰说明期望的操作和关键词
- 使用条件边配合关键词实现流程控制
- 考虑添加超时机制避免工作流无限等待

140
docs/user_guide/zh/nodes/literal.md Executable file
View File

@ -0,0 +1,140 @@
# Literal 节点
Literal 节点用于输出固定的文本内容。当节点被触发时，它会忽略所有输入，直接输出预定义的消息。
## 配置项
| 字段 | 类型 | 必填 | 默认值 | 说明 |
|------|------|------|--------|------|
| `content` | text | 是 | - | 输出的固定文本内容，不能为空 |
| `role` | string | 否 | `user` | 消息角色：`user` 或 `assistant` |
## 核心概念
### 固定输出
Literal 节点的特点：
- **忽略输入**：不管上游传入什么内容，都不影响输出
- **固定内容**：每次执行都输出相同的 `content`
- **角色标记**：输出消息带有指定的角色标识
### 消息角色
- `user`：表示这是用户发出的消息
- `assistant`：表示这是助手（AI）发出的消息
角色设置会影响下游节点对消息的处理方式。
## 何时使用
- **固定提示注入**：向流程中注入固定的指令或上下文
- **测试调试**：使用固定输入测试下游节点
- **默认响应**：在特定条件下返回固定消息
- **流程初始化**：作为工作流的起点提供初始内容
## 示例
### 基础用法
```yaml
nodes:
- id: Welcome Message
type: literal
config:
content: |
欢迎使用智能助手！请描述您的需求。
role: assistant
```
### 注入固定上下文
```yaml
nodes:
- id: Context Injector
type: literal
config:
content: |
请注意以下规则：
1. 回答必须简洁明了
2. 使用中文回复
3. 如有不确定，请说明
role: user
- id: Assistant
type: agent
config:
provider: openai
name: gpt-4o
edges:
- from: Context Injector
to: Assistant
```
### 条件分支中的固定响应
```yaml
nodes:
- id: Classifier
type: agent
config:
provider: openai
name: gpt-4o
role: 判断用户意图，回复 KNOWN 或 UNKNOWN
- id: Known Response
type: literal
config:
content: 我能帮助您完成这个任务。
role: assistant
- id: Unknown Response
type: literal
config:
content: 抱歉，我无法理解您的请求，请换一种方式描述。
role: assistant
edges:
- from: Classifier
to: Known Response
condition:
type: keyword
config:
any: [KNOWN]
- from: Classifier
to: Unknown Response
condition:
type: keyword
config:
any: [UNKNOWN]
```
### 测试用途
```yaml
nodes:
- id: Test Input
type: literal
config:
content: |
这是一段测试文本，用于验证下游处理逻辑。
包含多行内容。
role: user
- id: Processor
type: python
config:
timeout_seconds: 30
edges:
- from: Test Input
to: Processor
start: [Test Input]
```
## 注意事项
- `content` 字段不能为空字符串
- 使用 YAML 多行字符串语法 `|` 便于编写长文本
- 选择正确的 `role` 以确保下游节点正确处理消息

153
docs/user_guide/zh/nodes/loop_counter.md Executable file
View File

@ -0,0 +1,153 @@
# Loop Counter 节点
Loop Counter 节点是一种循环控制节点，用于限制工作流中环路的执行次数。它通过计数机制，在达到预设上限前抑制输出，达到上限后才释放消息触发出边，从而终止循环。
## 配置项
| 字段 | 类型 | 必填 | 默认值 | 说明 |
|------|------|------|--------|------|
| `max_iterations` | int | 是 | `10` | 最大循环次数，必须 ≥ 1 |
| `reset_on_emit` | bool | 否 | `true` | 达到上限后是否重置计数器 |
| `message` | text | 否 | - | 达到上限时发送给下游的消息内容 |
## 核心概念
### 工作原理
Loop Counter 节点维护一个内部计数器，其行为如下：
1. **每次被触发时**：计数器 +1
2. **计数器 < `max_iterations`**：**不产生任何输出**，出边不会被触发
3. **计数器 = `max_iterations`**：产生输出消息，触发出边
这种"抑制-释放"机制使得 Loop Counter 可以精确控制循环何时终止。
### 拓扑结构要求
Loop Counter 节点在图结构中有特殊的位置要求：
```
┌──────────────────────────────────────┐
▼ │
Agent ──► Human ─────► Loop Counter ──┬──┘
▲ │ │
└─────────┘ ▼
End Node (环外)
```
> **重要**：由于 Loop Counter **未达上限时不产生任何输出**，因此：
> - **Human 必须同时连接到 Agent 和 Loop Counter**：这样"继续循环"的边由 Human → Agent 承担，而 Loop Counter 仅负责计数
> - **Loop Counter 必须连接到 Agent（环内）**：使其被识别为环内节点，避免提前终止环路
> - **Loop Counter 必须连接到 End Node（环外）**：当达到上限时触发环外节点，终止整个环的执行
### 计数器状态
- 计数器状态在整个工作流执行期间持久化
- 当 `reset_on_emit: true` 时，达到上限后计数器重置为 0
- 当 `reset_on_emit: false` 时，达到上限后继续累计，后续每次触发都会输出
## 何时使用
- **防止无限循环**：为人机交互循环设置安全上限
- **迭代控制**：限制 Agent 自我迭代改进的最大轮次
- **超时保护**：作为流程执行的"熔断器"
## 示例
### 基础用法
```yaml
nodes:
- id: Iteration Guard
type: loop_counter
config:
max_iterations: 5
reset_on_emit: true
message: 已达到最大迭代次数，流程终止。
```
### 人机交互循环保护
这是 Loop Counter 最典型的使用场景：
```yaml
graph:
id: review_loop
description: 带迭代上限的审稿循环
nodes:
- id: Writer
type: agent
config:
provider: openai
name: gpt-4o
role: 根据用户反馈改进文章
- id: Reviewer
type: human
config:
description: |
审阅文章，输入 ACCEPT 接受或提供修改意见。
- id: Loop Guard
type: loop_counter
config:
max_iterations: 3
message: 已达到最大修改次数（3次），流程自动结束。
- id: Final Output
type: passthrough
config: {}
edges:
# 主循环：Writer -> Reviewer
- from: Writer
to: Reviewer
# 条件1：用户输入 ACCEPT -> 结束
- from: Reviewer
to: Final Output
condition:
type: keyword
config:
any: [ACCEPT]
# 条件2：用户输入修改意见 -> 同时触发 Writer 继续循环 AND Loop Guard 计数
- from: Reviewer
to: Writer
condition:
type: keyword
config:
none: [ACCEPT]
- from: Reviewer
to: Loop Guard
condition:
type: keyword
config:
none: [ACCEPT]
# Loop Guard 连接到 Writer（使其保持在环内）
- from: Loop Guard
to: Writer
# Loop Guard 达到上限时：触发 Final Output 结束流程
- from: Loop Guard
to: Final Output
start: [Writer]
end: [Final Output]
```
**执行流程说明**：
1. 用户首次输入修改意见 → 同时触发 Writer（继续循环）和 Loop Guard（计数 1，无输出）
2. 用户再次输入修改意见 → 同时触发 Writer（继续循环）和 Loop Guard（计数 2，无输出）
3. 用户第三次输入修改意见 → Writer 继续执行，Loop Guard 计数 3 达到上限，输出消息触发 Final Output，终止环路
4. 或者在任意时刻用户输入 ACCEPT → 直接到 Final Output 结束
## 注意事项
- `max_iterations` 必须为正整数（≥ 1）
- Loop Counter **未达上限时不产生任何输出**，出边不会触发
- 确保 Loop Counter 同时连接环内节点和环外节点
- `message` 字段可选，默认消息为 `"Loop limit reached (N)"`

206
docs/user_guide/zh/nodes/passthrough.md Executable file
View File

@ -0,0 +1,206 @@
# Passthrough 节点
Passthrough 节点是最简单的节点类型，它不执行任何操作，仅将接收到的消息传递给下游节点。默认情况下只传递**最后一条消息**。它主要用于图结构的"理线"优化和上下文控制。
## 配置项
| 字段 | 类型 | 必填 | 默认值 | 说明 |
|------|------|------|--------|------|
| `only_last_message` | bool | 否 | `true` | 是否只传递最后一条消息。设为 `false` 时传递所有消息。 |
### 基本配置
```yaml
config: {} # 使用默认配置，只传递最后一条消息
```
### 传递所有消息
```yaml
config:
only_last_message: false # 传递所有接收到的消息
```
## 核心概念
### 透传行为
- 接收上游传入的所有消息
- **默认只传递最后一条消息**（`only_last_message: true`）
- 设置 `only_last_message: false` 时传递所有消息
- 不做任何内容处理或转换
### 图结构优化
Passthrough 节点的核心价值不在于数据处理，而在于**图结构的优化**（"理线"）：
- 使复杂的边连接更加清晰
- 集中管理出边配置（如 `keep_message`）
- 作为逻辑分界点，提高工作流可读性
## 关键用途
### 1. 作为起始节点保留初始上下文
将 Passthrough 作为工作流的入口节点，配合边的 `keep_message: true` 配置，可以确保用户的初始任务始终保留在上下文中，不会被后续节点的输出覆盖：
```yaml
nodes:
- id: Task Keeper
type: passthrough
config: {}
- id: Worker A
type: agent
config:
provider: openai
name: gpt-4o
- id: Worker B
type: agent
config:
provider: openai
name: gpt-4o
edges:
# 从入口分发任务，保留原始消息
- from: Task Keeper
to: Worker A
keep_message: true # 保留初始任务上下文
- from: Task Keeper
to: Worker B
keep_message: true
start: [Task Keeper]
```
**效果**：Worker A 和 Worker B 都能看到用户的原始输入，而不仅仅是上一个节点的输出。
### 2. 过滤循环中的冗余输出
在包含循环的工作流中，循环内的节点可能产生大量中间输出。如果将所有输出都传递给后续节点，会导致上下文膨胀。使用 Passthrough 节点可以**只传递循环的最终结果**：
```yaml
nodes:
- id: Iterative Improver
type: agent
config:
provider: openai
name: gpt-4o
role: 根据反馈不断改进输出
- id: Evaluator
type: agent
config:
provider: openai
name: gpt-4o
role: |
评估输出质量，回复 GOOD 或提供改进建议
- id: Result Filter
type: passthrough
config: {}
- id: Final Processor
type: agent
config:
provider: openai
name: gpt-4o
role: 对最终结果进行后处理
edges:
- from: Iterative Improver
to: Evaluator
# 循环：评估不通过时回到改进节点
- from: Evaluator
to: Iterative Improver
condition:
type: keyword
config:
none: [GOOD]
# 循环结束：通过 Passthrough 过滤，只传递最后一条
- from: Evaluator
to: Result Filter
condition:
type: keyword
config:
any: [GOOD]
- from: Result Filter
to: Final Processor
start: [Iterative Improver]
end: [Final Processor]
```
**效果**：无论循环迭代多少次，`Final Processor` 只会收到 `Evaluator` 的最后一条输出（表示质量通过的那条），而不是所有中间结果。
## 其他用途
- **占位符**：在设计阶段预留节点位置
- **条件分支**：配合条件边实现路由逻辑
- **调试观察点**：在流程中插入便于观察的节点
## 示例
### 基础用法
```yaml
nodes:
- id: Router
type: passthrough
config: {}
```
### 条件路由
```yaml
nodes:
- id: Classifier
type: agent
config:
provider: openai
name: gpt-4o
role: |
分类输入内容，回复 TECHNICAL 或 BUSINESS
- id: Router
type: passthrough
config: {}
- id: Tech Handler
type: agent
config:
provider: openai
name: gpt-4o
- id: Biz Handler
type: agent
config:
provider: openai
name: gpt-4o
edges:
- from: Classifier
to: Router
- from: Router
to: Tech Handler
condition:
type: keyword
config:
any: [TECHNICAL]
- from: Router
to: Biz Handler
condition:
type: keyword
config:
any: [BUSINESS]
```
## 最佳实践
- 使用有意义的节点 ID 描述其拓扑作用（如 `Task Keeper`、`Result Filter`）
- 作为入口节点时，出边配置 `keep_message: true` 保留上下文
- 在循环后使用，可以过滤掉冗余的中间输出

89
docs/user_guide/zh/nodes/python.md Executable file
View File

@ -0,0 +1,89 @@
# Python 节点
Python 节点用于在工作流中执行 Python 脚本或内联代码，实现自定义数据处理、API 调用、文件操作等逻辑。脚本在共享的 `code_workspace/` 目录中执行，可访问工作流上下文数据。
## 配置项
| 字段 | 类型 | 必填 | 默认值 | 说明 |
|------|------|------|--------|------|
| `interpreter` | string | 否 | 当前 Python | Python 解释器路径 |
| `args` | list[str] | 否 | `[]` | 追加到解释器后的启动参数 |
| `env` | dict[str, str] | 否 | `{}` | 额外环境变量，会覆盖系统默认值 |
| `timeout_seconds` | int | 否 | `60` | 脚本执行超时时间（秒） |
| `encoding` | string | 否 | `utf-8` | 解析 stdout/stderr 的编码 |
## 核心概念
### 代码工作区
Python 脚本在 `code_workspace/` 目录下执行：
- 脚本可以读写该目录中的文件
- 多个 Python 节点共享同一工作区
- 工作区在单次工作流执行期间持久化
### 输入输出
- **输入**：上游节点的输出作为环境变量或标准输入传递
- **输出**：脚本的 stdout 输出将作为 Message 传递给下游节点
## 何时使用
- **数据处理**：解析 JSON/XML、数据转换、格式化
- **API 调用**：调用第三方服务、获取外部数据
- **文件操作**：读写文件、生成报告
- **复杂计算**：数学运算、算法实现
- **胶水逻辑**：连接不同节点的自定义逻辑
## 示例
### 基础配置
```yaml
nodes:
- id: Data Processor
type: python
config:
timeout_seconds: 120
env:
key: value
```
### 指定解释器和参数
```yaml
nodes:
- id: Script Runner
type: python
config:
interpreter: /usr/bin/python3.11
timeout_seconds: 300
encoding: utf-8
```
### 典型工作流示例
```yaml
nodes:
- id: LLM Generator
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
role: 你需要根据用户的输入，生成可执行的 Python 代码。代码应当包裹在 ```python ``` 之间。
- id: Result Parser
type: python
config:
timeout_seconds: 30
edges:
- from: LLM Generator
to: Result Parser
```
## 注意事项
- 确保脚本文件放置在 `code_workspace/` 目录下
- 长时间运行的脚本应适当增加 `timeout_seconds`
- 使用 `env` 传递额外的环境变量，可在脚本中通过 `os.getenv` 访问

138
docs/user_guide/zh/nodes/subgraph.md Executable file
View File

@ -0,0 +1,138 @@
# Subgraph 节点
Subgraph 节点允许将另一个工作流图嵌入到当前工作流中，实现流程复用和模块化设计。子图可以来自外部 YAML 文件，也可以直接在配置中内联定义。
## 配置项
| 字段 | 类型 | 必填 | 默认值 | 说明 |
|------|------|------|--------|------|
| `type` | string | 是 | - | 子图来源类型：`file` 或 `config` |
| `config` | object | 是 | - | 根据 `type` 不同，包含不同的配置 |
### file 类型配置
| 字段 | 类型 | 必填 | 说明 |
|------|------|------|------|
| `path` | string | 是 | 子图文件路径（相对于 `yaml_instance/` 或绝对路径） |
### config 类型配置
内联定义完整的子图结构，包含与顶层 `graph` 相同的字段：
| 字段 | 类型 | 必填 | 说明 |
|------|------|------|------|
| `id` | string | 是 | 子图标识符 |
| `description` | string | 否 | 子图描述 |
| `log_level` | string | 否 | 日志级别（DEBUG/INFO） |
| `nodes` | list | 是 | 节点列表 |
| `edges` | list | 否 | 边列表 |
| `start` | list | 否 | 入口节点列表 |
| `end` | list | 否 | 出口节点列表 |
| `memory` | list | 否 | 子图专用的 Memory 定义 |
## 核心概念
### 模块化复用
将常用的流程片段抽取为独立的 YAML 文件，多个工作流可以复用同一子图：
- 例如：将"文章润色"流程封装为子图
- 不同的主工作流都可以调用该子图
### 变量继承
子图会继承父图的 `vars` 变量定义，支持跨层级变量传递。
### 执行隔离
子图作为独立单元执行，拥有自己的：
- 节点命名空间
- 日志级别配置
- Memory 定义（可选）
## 何时使用
- **流程复用**：多个工作流共享相同的子流程
- **模块化设计**：将复杂流程拆分为可管理的小单元
- **团队协作**：不同团队维护不同的子图模块
## 示例
### 引用外部文件
```yaml
nodes:
- id: Review Process
type: subgraph
config:
type: file
config:
path: common/review_flow.yaml
```
### 内联定义子图
```yaml
nodes:
- id: Translation Unit
type: subgraph
config:
type: config
config:
id: translation_subgraph
description: 多语言翻译子流程
nodes:
- id: Translator
type: agent
config:
provider: openai
name: gpt-4o
role: 你是一位专业翻译，将内容翻译为目标语言。
- id: Proofreader
type: agent
config:
provider: openai
name: gpt-4o
role: 你是一位校对专家，检查并润色翻译内容。
edges:
- from: Translator
to: Proofreader
start: [Translator]
end: [Proofreader]
```
### 组合多个子图
```yaml
nodes:
- id: Input Handler
type: agent
config:
provider: openai
name: gpt-4o
- id: Analysis Module
type: subgraph
config:
type: file
config:
path: modules/analysis.yaml
- id: Report Module
type: subgraph
config:
type: file
config:
path: modules/report_gen.yaml
edges:
- from: Input Handler
to: Analysis Module
- from: Analysis Module
to: Report Module
```
## 注意事项
- 子图文件路径支持相对路径（基于 `yaml_instance/`）和绝对路径
- 避免循环嵌套（A 引用 B，B 再引用 A）
- 子图的 `start` 和 `end` 节点决定了数据如何流入流出，这决定了子图如何处理父图传入的消息，以及以哪个节点的最终输出作为返回给父图的消息。

99
docs/user_guide/zh/web_ui_guide.md Executable file
View File

@ -0,0 +1,99 @@
# 前端 Web UI 快速入门指南
本指南帮助用户快速上手 DevAll Web UI，涵盖主要功能页面和操作流程。
## 1. 系统入口
启动前后端服务后，访问 `http://localhost:5173` 进入 Web UI。
## 2. 主要页面
### 2.1 首页 (Home)
系统首页，提供快速导航入口。
### 2.2 工作流列表 (Workflow List)
查看和管理所有可用的工作流 YAML 文件。
**功能**：
- 浏览 `yaml_instance/` 目录下的工作流
- 预览 YAML 配置内容
- 选择工作流进入执行或编辑
### 2.3 启动页 (Launch View)
工作流执行的主界面，是最常用的页面。
**操作流程**：
1. **选择工作流**：从左侧列表选择要执行的 YAML 文件
2. **上传附件**（可选）：点击上传按钮添加文件（如 CSV 数据、图片等）
3. **输入任务提示**：在文本框中输入指导工作流执行的提示词
4. **点击 Launch**：启动工作流执行
**执行期间**：
- **节点视图**：观察节点状态变化（pending → running → success/failed）
- **输出面板**：实时查看执行日志、节点输出上下文和生成的工件（三者共用同一面板）
**人工输入**：
- 当执行到 `human` 节点时，界面会显示输入提示
- 填写文本内容或上传附件后提交继续执行
### 2.4 工作流工作台 (Workflow Workbench)
可视化工作流编辑器。
**功能**：
- 拖拽式节点编辑
- 节点配置面板
- 边连接与条件设置
- 导出为 YAML 文件
### 2.5 教程页 (Tutorial)
内置教程，帮助新用户了解系统功能。
## 3. 常用操作
### 3.1 运行工作流
1. 进入 **Launch View**
2. 从左侧选择工作流
3. 输入 Task Prompt（任务提示）
4. 点击 **Launch** 按钮
5. 观察执行过程，等待完成
### 3.2 下载运行结果
执行完成后：
1. 点击右侧面板的 **Download** 按钮
2. 下载完整的 Session 压缩包（含 context.json、附件、日志等）
### 3.3 人工审阅节点交互
当工作流包含 `human` 节点时：
1. 执行暂停并显示提示信息
2. 阅读上下文内容
3. 输入审阅意见或操作指令
4. 点击提交继续执行
## 4. 快捷键
| 快捷键 | 功能 |
|--------|------|
| `Ctrl/Cmd + Enter` | 提交输入 |
| `Esc` | 关闭弹窗/面板 |
## 5. 故障排查
| 问题 | 解决方案 |
|------|----------|
| 页面无法加载 | 确认前端服务 `npm run dev` 正常运行 |
| 无法连接后端 | 确认后端服务 `uv run python server_main.py` 正常运行 |
| 工作流列表为空 | 检查 `yaml_instance/` 目录是否有 YAML 文件 |
| 执行无响应 | 检查浏览器开发者工具的 Network/Console 日志 |
| WebSocket 断开 | 刷新页面重新建立连接 |
## 6. 相关文档
- [工作流编排](workflow_authoring.md) - YAML 编写指南

247
docs/user_guide/zh/workflow_authoring.md Executable file
View File

@ -0,0 +1,247 @@
# 工作流编排指南
本指南聚焦 YAML 结构、节点类型、Provider 配置、边条件与模板导出，帮助工作流作者快速构建与调试 DAG。
## 1. 必备背景
- 熟悉 `yaml_instance/` 与 `yaml_template/` 的目录结构。
- 了解基本节点类型（`model`、`python`、`agent`、`human`、`subgraph`、`passthrough`、`literal`）。
- 理解 `FIELD_SPECS`（见 [field_specs.md](field_specs.md)）与 Schema API（见 [config_schema_contract.md](config_schema_contract.md)）可被前端/IDE 用于动态表单。
## 2. YAML 顶层结构
所有工作流文件都遵循 `DesignConfig` 根结构，仅包含 `version`、`vars`、`graph` 三个键。下面示例节选自 `yaml_instance/net_example.yaml`，可以直接运行：
```yaml
version: 0.4.0
vars:
BASE_URL: https://api.example.com/v1
API_KEY: ${API_KEY}
graph:
id: paper_gen
description: 文章生成与润色
log_level: INFO
is_majority_voting: false
initial_instruction: |
这是一个文章生成与润色流程，请输入一个词语或短句作为任务提示。
start:
- Article Writer
end:
- Article Writer
nodes:
- id: Article Writer
type: agent
config:
provider: openai
base_url: ${BASE_URL}
api_key: ${API_KEY}
name: gpt-4o
params:
temperature: 0.1
- id: Human Reviewer
type: human
config:
description: 请审阅文章，如接受结果请输入 ACCEPT 结束流程；否则输入修改意见。
edges:
- from: Article Writer
to: Human Reviewer
- from: Human Reviewer
to: Article Writer
condition:
type: keyword
config:
none:
- ACCEPT
case_sensitive: false
```
- `version`：配置版本号，缺省为 `0.0.0`。当 `entity/configs/graph.py` 中的 Schema 发生破坏性调整时，用于与前端模板和迁移脚本对齐。
- `vars`：根级键值对，可在任意字段使用 `${VAR}` 占位，若未命中则回退到同名环境变量。`GraphDefinition.from_dict` 会拒绝在子图或节点下声明 `vars`，因此请仅在顶层维护。
**环境变量与 `.env` 文件**
系统支持在 YAML 配置中使用 `${VAR}` 语法引用变量。这些变量可用于配置中的任意字符串字段，常见用途包括：
- **API 密钥**：`api_key: ${API_KEY}`
- **服务地址**：`base_url: ${BASE_URL}`
- **模型名称**：`name: ${MODEL_NAME}`
系统在解析配置时会自动加载项目根目录下的 `.env` 文件（若存在）。变量解析的优先级如下：
| 优先级 | 来源 | 说明 |
| --- | --- | --- |
| 1（最高） | `vars` 中显式定义的值 | YAML 文件中直接声明的键值对 |
| 2 | 系统/Shell 环境变量 | 如通过 `export` 设置的值 |
| 3（最低） | `.env` 文件中的值 | 仅当环境变量尚未存在时生效 |
> [!TIP]
> `.env` 文件不会覆盖已存在的环境变量。这意味着您可以在 `.env` 中定义默认值，同时通过 `export` 或部署平台的环境变量配置来覆盖它们。
> [!WARNING]
> 若占位符引用的变量在上述三个来源中均未定义，配置解析时将抛出 `ConfigError` 并指明出错路径。
- `graph`：唯一必填段落，映射到 `GraphDefinition` dataclass。它包含：
- **基础元信息**：`id`（必填）、`description`、`log_level`（默认 `DEBUG`）、`is_majority_voting`、`initial_instruction`、可选 `organization`。
- **执行控制**：`start`/`end`（入口出口列表；系统会在启动时执行 `start` 中的节点）、`nodes`、`edges`。`nodes` 与 `edges` 同步 `entity/configs/node/*.py` 与 `entity/configs/edge.py`，所有 Provider、模型、Tooling 配置都挂在 `node.config` 内，不再在顶层维护 `providers` 表。上例通过 `keyword` 条件在 `Human Reviewer -> Article Writer` 边上避免输入 `ACCEPT` 时继续循环。
- **共享资源**：`memory`（定义 Memory store 列表，供模型节点的 `config.memories` 引用）。调度器会校验节点引用是否在 `graph.memory` 中声明。
- **Schema 参考**：`yaml_template/design.yaml` 会实时反映 `GraphDefinition` 字段，建议在修改后运行 `python -m tools.export_design_template` 或调用 Schema API 校验。
进一步阅读：`docs/user_guide/zh/field_specs.md`（字段精细描述）、`docs/user_guide/zh/runtime_ops.md`（运行期可观测性）、以及 `yaml_template/design.yaml`（自动生成的基准模板）。
## 3. 节点类型速览
| 类型 | 描述 | 关键字段 | 详细文档 |
| --- |------------------------------------------| --- | --- |
| `agent` | 调用 LLM，支持工具、记忆、thinking | `provider`, `model`, `prompt_template`, `tooling`, `thinking`, `memories` | [agent.md](nodes/agent.md) |
| `python` | 执行 Python 代码（脚本或指令），共享 `code_workspace/` | `entry_script`, `inline_code`, `timeout`, `env` | [python.md](nodes/python.md) |
| `human` | 在 Web UI 阻塞等待人工输入 | `prompt`, `timeout`, `attachments` | [human.md](nodes/human.md) |
| `subgraph` | 嵌入子 DAG，复用复杂流程 | `graph_path` 或内联 `graph` | [subgraph.md](nodes/subgraph.md) |
| `passthrough` | 透传节点，默认只传递最后一条消息，可传递所有信息；用于上下文过滤和图结构优化 | `only_last_message` | [passthrough.md](nodes/passthrough.md) |
| `literal` | 被触发时输出固定文本消息，忽略输入 | `content`, `role`（`user`/`assistant`） | [literal.md](nodes/literal.md) |
| `loop_counter` | 限制环路执行次数的控制节点 | `max_iterations`, `reset_on_emit`, `message` | [loop_counter.md](nodes/loop_counter.md) |
详细字段可在前端使用 Schema API (`POST /api/config/schema`) 动态查询，也可参照 `entity/configs/` 中同名 dataclass。
## 4. Provider 与 Agent 设置
- `provider` 字段缺省时，使用 `globals.default_provider`（如 `openai`）。
- `model`、`api_key`、`base_url` 等字段支持 `${VAR}` 占位，便于跨环境复用。
- 对接多个 Provider 时，可在 workflow 层设置 `globals`: `{ default_provider: ..., retry: {...} }`（若 dataclass 支持）。
### 4.1 Gemini Provider 配置示例
```yaml
model:
provider: gemini
base_url: https://generativelanguage.googleapis.com
api_key: ${GEMINI_API_KEY}
name: gemini-2.0-flash-001
input_mode: messages
params:
response_modalities: ["text", "image"]
safety_settings:
- category: HARM_CATEGORY_SEXUAL
threshold: BLOCK_LOWER
```
Gemini Provider 支持多模态输入（图片/视频/音频会自动转换为 Part），并支持 `function_calling_config` 来控制工具调用行为。
## 5. 边与条件
- 基本边：
```yaml
- source: plan
target: execute
```
- 条件边：
```yaml
edges:
- source: router
target: analyze
condition:
type: function
config:
name: should_analyze # functions/edge/should_analyze.py
```
- 当 `condition` 抛错时，调度器会记录错误并抛出 `WorkflowExecutionError`，导致该分支（通常是整个运行）终止，后继节点不会继续执行。
- 通过注册中心可以声明更多条件类型，例如内置的 `keyword`（无需写 Python 函数）：
```yaml
edges:
- from: review
to: finalize
condition:
type: keyword
config:
any: ["FINAL", "APPROVED"]
none: ["RETRY"]
case_sensitive: false # 默认为 true
```
`condition.type` 的合法值由后端注册中心（使用 `register_edge_condition` 注册）决定，schema 会自动在前端的下拉列表中展示 `summary` 描述。默认的 `function` 类型兼容旧写法（直接填写函数名字符串），未提供配置时等价于 `name: true`。
### 5.1 边级 Payload Processor
- 场景：当条件成立后希望“先处理一下消息”，例如根据正则提取得分、只保留结构化字段或者调用自定义函数对文本重写。
- YAML 字段：在任意边上新增 `process`，结构与 `condition` 相同（`type + config`），目前内置
- `regex_extract`：基于 Python 正则。支持 `pattern`、`group`（名称或序号）、`mode`（`replace_content`、`metadata`、`data_block`）、`multiple`、`on_no_match`（`pass`/`default`/`drop`）等字段。
- `function`：调用 `functions/edge_processor/*.py` 中的处理函数。函数签名为 `def foo(payload: Message, **kwargs) -> Message | None`。现在Processor 接口已标准化，`kwargs` 中包含了 `context: ExecutionContext`，可访问当前执行上下文。
- 运行时行为：
- Processor 在条件通过且 `carry_data=true` 时执行，若返回 `None`，该边不会触发也不会向后继节点发送输入。
- 日志中会在 `EDGE_PROCESS` 事件里显示 `process_label`、`process_type`，便于排查。
- 示例：
```yaml
edges:
- from: reviewer
to: qa
process:
type: regex_extract
config:
pattern: "Score\\s*:\\s*(?P<score>\\d+)"
group: score
mode: metadata
metadata_key: "quality_score"
case_sensitive: false
on_no_match: default
default_value: "0"
```
## 6. 模型节点高级特性
- **Tooling**：在 `AgentConfig.tooling` 中配置，具体见 [Tooling 模块](modules/tooling/README.md)。
- **Thinking**：在 `AgentConfig.thinking` 中开启，如 `chain-of-thought`、`reflection`（详见 `entity/configs/thinking.py`）。
- **Memories**：`AgentConfig.memories` 绑定 `MemoryAttachmentConfig`，详见 [Memory 模块](modules/memory.md)。
## 7. 动态执行 (Map-Reduce/Tree)
节点配置新增同级字段 `dynamic`，用于启用并行处理或 Map-Reduce 模式。
### 7.1 核心概念
- **Map 模式** (`type: map`)：扇出（Fan-out）。将 List 输入拆分为多个单元并行执行，输出 `List[Message]`（结果打平）。
- **Tree 模式** (`type: tree`)：扇出与归约（Fan-out & Reduce）。将输入拆分并行执行后，按 `group_size` 分组递归归约，最终输出单个结果（如“总结的总结”）。
- **Split 策略**：定义如何将上一节点的输出或当前输入拆分为并行单元。
### 7.2 配置结构
```yaml
nodes:
- id: Research Agents
type: agent
# 常规配置（作为并行单元的模板）
config:
provider: openai
model: gpt-4o
prompt_template: "Research this topic: {{content}}"
# 动态执行配置
dynamic:
type: map
# 拆分策略 (仅首层有效)
split:
type: message # 可选: message, regex, json_path
# pattern: "..." # regex 模式下必填
# json_path: "$.items[*]" # json_path 模式下必填
# 模式专属配置
config:
max_parallel: 5 # 控制并发度
```
### 7.3 Tree 模式示例
适用于长文本分段摘要等场景：
```yaml
dynamic:
type: tree
split:
type: regex
pattern: "(?s).{1,2000}(?:\\s|$)" # 每 2000 字符切分
config:
group_size: 3 # 每 3 个结果归约为 1 个
max_parallel: 10
```
该模式会自动构建多层级执行树，直到结果数量归约为 1。split 配置与 map 模式一致，
## 8. 设计模板导出
任意修改 Config/FIELD_SPECS 后，运行：
```bash
python -m tools.export_design_template \
--output yaml_template/design.yaml \
--mirror frontend/public/design_0.4.0.yaml
```
- 命令会读取注册表（节点、memory、tooling 等）与 `FIELD_SPECS`，自动生成 YAML 模板与前端镜像。
- 更新后请提交模板文件，并通知前端刷新静态资源。
## 9. CLI / API 运行
- **Web UI**：访问前端页面 → 选择 YAML → 填写运行参数 → 启动 → 在面板监控。**我们建议您采用此方式运行。**
- **HTTP**：`POST /api/workflow/execute`，payload 包含 `session_name`, `graph_path` 或 `graph_content`, `task_prompt`、可选的 `attachments`，以及 `log_level`（默认 `INFO`，支持 `INFO` 或 `DEBUG`）。
- **CLI**：`python run.py --path yaml_instance/demo.yaml --name test_run`（执行前可设置 `TASK_PROMPT` 环境变量或在 CLI 提示中输入）。
## 10. 调试建议
- 使用 Web UI 的上下文快照或 WareHouse 中的 `context.json` 检查节点输入输出。注意所有节点输出现已统一为 `List[Message]` 结构。
- 结合 [config_schema_contract.md](config_schema_contract.md) 的 breadcrumbs 功能，用 CLI `python run.py --inspect-schema` 快速查看字段定义。
- 若 YAML 占位符缺失，解析阶段会抛出 `ConfigError`，在 UI/CLI 中都可看到明确路径。

0
entity/__init__.py Executable file
View File

33
entity/config_loader.py Executable file
View File

@ -0,0 +1,33 @@
"""Helpers for loading validated configuration objects."""
from pathlib import Path
from typing import Any, Mapping
import yaml
from entity.configs import DesignConfig, ConfigError
from utils.env_loader import load_dotenv_file, build_env_var_map
from utils.vars_resolver import resolve_design_placeholders
def prepare_design_mapping(data: Mapping[str, Any], *, source: str | None = None) -> Mapping[str, Any]:
load_dotenv_file()
env_lookup = build_env_var_map()
prepared = dict(data)
resolve_design_placeholders(prepared, env_lookup=env_lookup, path=source or "root")
return prepared
def load_design_from_mapping(data: Mapping[str, Any], *, source: str | None = None) -> DesignConfig:
"""Parse a raw dictionary into a typed :class:`DesignConfig`."""
prepared = prepare_design_mapping(data, source=source)
return DesignConfig.from_dict(prepared, path="root")
def load_design_from_file(path: Path) -> DesignConfig:
"""Read a YAML file and parse it into a :class:`DesignConfig`."""
with path.open("r", encoding="utf-8") as handle:
data = yaml.load(handle, Loader=yaml.FullLoader)
if not isinstance(data, Mapping):
raise ConfigError("YAML root must be a mapping", path=str(path))
return load_design_from_mapping(data, source=str(path))

54
entity/configs/__init__.py Executable file
View File

@ -0,0 +1,54 @@
"""Configuration package exports."""
from .base import BaseConfig, ConfigError
from .edge.edge import EdgeConfig
from .edge.edge_condition import EdgeConditionConfig, FunctionEdgeConditionConfig, KeywordEdgeConditionConfig
from .edge.edge_processor import EdgeProcessorConfig, RegexEdgeProcessorConfig, FunctionEdgeProcessorConfig
from .graph import DesignConfig, GraphDefinition
from .node.memory import (
BlackboardMemoryConfig,
EmbeddingConfig,
FileMemoryConfig,
FileSourceConfig,
MemoryAttachmentConfig,
MemoryStoreConfig,
SimpleMemoryConfig,
)
from .node.agent import AgentConfig, AgentRetryConfig
from .node.human import HumanConfig
from .node.subgraph import SubgraphConfig
from .node.node import EdgeLink, Node
from .node.passthrough import PassthroughConfig
from .node.python_runner import PythonRunnerConfig
from .node.thinking import ReflectionThinkingConfig, ThinkingConfig
from .node.tooling import FunctionToolConfig, McpLocalConfig, McpRemoteConfig, ToolingConfig
__all__ = [
"AgentConfig",
"AgentRetryConfig",
"BaseConfig",
"ConfigError",
"DesignConfig",
"EdgeConfig",
"EdgeConditionConfig",
"EdgeLink",
"EdgeProcessorConfig",
"RegexEdgeProcessorConfig",
"FunctionEdgeProcessorConfig",
"BlackboardMemoryConfig",
"EmbeddingConfig",
"FileSourceConfig",
"FunctionToolConfig",
"GraphDefinition",
"HumanConfig",
"MemoryAttachmentConfig",
"MemoryStoreConfig",
"McpLocalConfig",
"McpRemoteConfig",
"Node",
"PassthroughConfig",
"PythonRunnerConfig",
"SubgraphConfig",
"ThinkingConfig",
"ToolingConfig",
]

276
entity/configs/base.py Executable file
View File

@ -0,0 +1,276 @@
"""Shared helpers and base classes for configuration dataclasses."""
from dataclasses import dataclass, field, replace
from typing import Any, Dict, Iterable, List, Mapping, MutableMapping, Sequence, TypeVar, ClassVar, Optional
TConfig = TypeVar("TConfig", bound="BaseConfig")
class ConfigError(ValueError):
"""Raised when configuration parsing or validation fails."""
def __init__(self, message: str, path: str | None = None):
self.path = path
full_message = f"{path}: {message}" if path else message
super().__init__(full_message)
@dataclass(frozen=True)
class RuntimeConstraint:
"""Represents a conditional requirement for configuration fields."""
when: Mapping[str, Any]
require: Sequence[str]
message: str
def to_json(self) -> Dict[str, Any]:
return {
"when": dict(self.when),
"require": list(self.require),
"message": self.message,
}
@dataclass(frozen=True)
class ChildKey:
"""Identifies a conditional navigation target for nested schemas."""
field: str
value: Any | None = None
# variant: str | None = None
def matches(self, field: str, value: Any | None) -> bool:
if self.field != field:
return False
# if self.variant is not None and self.variant != str(value):
# return False
if self.value is None:
return True
return self.value == value
def to_json(self) -> Dict[str, Any]:
payload: Dict[str, Any] = {"field": self.field}
if self.value is not None:
payload["value"] = self.value
# if self.variant is not None:
# payload["variant"] = self.variant
return payload
@dataclass(frozen=True)
class EnumOption:
"""Rich metadata for enum values shown in UI."""
value: Any
label: str | None = None
description: str | None = None
def to_json(self) -> Dict[str, Any]:
payload: Dict[str, Any] = {"value": self.value}
if self.label:
payload["label"] = self.label
if self.description:
payload["description"] = self.description
return payload
@dataclass(frozen=True)
class ConfigFieldSpec:
"""Describes a single configuration field for schema export."""
name: str
type_hint: str
required: bool = False
display_name: str | None = None
default: Any | None = None
enum: Sequence[Any] | None = None
enum_options: Sequence[EnumOption] | None = None
description: str | None = None
child: type["BaseConfig"] | None = None
advance: bool = False
# ui: Mapping[str, Any] | None = None
def with_name(self, name: str) -> "ConfigFieldSpec":
if self.name == name:
return self
return replace(self, name=name)
def to_json(self) -> Dict[str, Any]:
display = self.display_name or self.name
data: Dict[str, Any] = {
"name": self.name,
"displayName": display,
"type": self.type_hint,
"required": self.required,
"advance": self.advance,
}
if self.default is not None:
data["default"] = self.default
if self.enum is not None:
data["enum"] = list(self.enum)
if self.enum_options:
data["enumOptions"] = [option.to_json() for option in self.enum_options]
if self.description:
data["description"] = self.description
if self.child is not None:
data["childNode"] = self.child.__name__
# if self.ui:
# data["ui"] = dict(self.ui)
return data
@dataclass(frozen=True)
class SchemaNode:
"""Serializable representation of a configuration node."""
node: str
fields: Sequence[ConfigFieldSpec]
constraints: Sequence[RuntimeConstraint] = field(default_factory=list)
def to_json(self) -> Dict[str, Any]:
return {
"node": self.node,
"fields": [spec.to_json() for spec in self.fields],
"constraints": [constraint.to_json() for constraint in self.constraints],
}
@dataclass
class BaseConfig:
"""Base dataclass providing validation and schema hooks."""
path: str
# Class-level hooks populated by concrete configs.
FIELD_SPECS: ClassVar[Dict[str, ConfigFieldSpec]] = {}
CONSTRAINTS: ClassVar[Sequence[RuntimeConstraint]] = ()
CHILD_ROUTES: ClassVar[Dict[ChildKey, type["BaseConfig"]]] = {}
def __post_init__(self) -> None: # pragma: no cover - thin wrapper
self.validate()
def validate(self) -> None:
"""Hook for subclasses to implement structural validation."""
# Default implementation intentionally empty.
return None
@classmethod
def field_specs(cls) -> Dict[str, ConfigFieldSpec]:
return {name: spec.with_name(name) for name, spec in getattr(cls, "FIELD_SPECS", {}).items()}
@classmethod
def constraints(cls) -> Sequence[RuntimeConstraint]:
return tuple(getattr(cls, "CONSTRAINTS", ()) or ())
@classmethod
def child_routes(cls) -> Dict[ChildKey, type["BaseConfig"]]:
return dict(getattr(cls, "CHILD_ROUTES", {}) or {})
@classmethod
def resolve_child(cls, field: str, value: Any | None = None) -> type["BaseConfig"] | None:
for key, target in cls.child_routes().items():
if key.matches(field, value):
return target
return None
def as_config(self, expected_type: type[TConfig], *, attr: str = "config") -> TConfig | None:
"""Return the nested config stored under *attr* if it matches the expected type."""
value = getattr(self, attr, None)
if isinstance(value, expected_type):
return value
return None
@classmethod
def collect_schema(cls) -> SchemaNode:
return SchemaNode(node=cls.__name__, fields=list(cls.field_specs().values()), constraints=list(cls.constraints()))
@classmethod
def example(cls) -> Dict[str, Any]:
"""Placeholder for future example export support."""
return {}
T = TypeVar("T")
def ensure_list(value: Any) -> List[Any]:
if value is None:
return []
if isinstance(value, list):
return list(value)
if isinstance(value, (tuple, set)):
return list(value)
return [value]
def ensure_dict(value: Mapping[str, Any] | None) -> Dict[str, Any]:
if value is None:
return {}
if isinstance(value, MutableMapping):
return dict(value)
if isinstance(value, Mapping):
return dict(value)
raise ConfigError("expected mapping", path=str(value))
def require_mapping(data: Any, path: str) -> Mapping[str, Any]:
if not isinstance(data, Mapping):
raise ConfigError("expected mapping", path)
return data
def require_str(data: Mapping[str, Any], key: str, path: str, *, allow_empty: bool = False) -> str:
value = data.get(key)
key_path = f"{path}.{key}" if path else key
if not isinstance(value, str):
raise ConfigError("expected string", key_path)
if not allow_empty and not value.strip():
raise ConfigError("expected non-empty string", key_path)
return value
def optional_str(data: Mapping[str, Any], key: str, path: str) -> str | None:
value = data.get(key)
if value is None or value == "":
return None
key_path = f"{path}.{key}" if path else key
if not isinstance(value, str):
raise ConfigError("expected string", key_path)
return value
def require_bool(data: Mapping[str, Any], key: str, path: str) -> bool:
value = data.get(key)
key_path = f"{path}.{key}" if path else key
if not isinstance(value, bool):
raise ConfigError("expected boolean", key_path)
return value
def optional_bool(data: Mapping[str, Any], key: str, path: str, *, default: bool | None = None) -> bool | None:
if key not in data:
return default
value = data[key]
key_path = f"{path}.{key}" if path else key
if not isinstance(value, bool):
raise ConfigError("expected boolean", key_path)
return value
def optional_dict(data: Mapping[str, Any], key: str, path: str) -> Dict[str, Any] | None:
if key not in data or data[key] is None:
return None
value = data[key]
key_path = f"{path}.{key}" if path else key
if not isinstance(value, Mapping):
raise ConfigError("expected mapping", key_path)
return dict(value)
def extend_path(path: str, suffix: str) -> str:
if not path:
return suffix
if suffix.startswith("["):
return f"{path}{suffix}"
return f"{path}.{suffix}"

443
entity/configs/dynamic_base.py Executable file
View File

@ -0,0 +1,443 @@
"""Shared dynamic configuration classes for both node and edge level execution.
This module contains the base classes used by both node-level and edge-level
dynamic execution configurations to avoid circular imports.
"""
from dataclasses import dataclass, fields, replace
from typing import Any, ClassVar, Dict, Mapping, Optional, Type, TypeVar
from entity.configs.base import (
BaseConfig,
ChildKey,
ConfigError,
ConfigFieldSpec,
extend_path,
optional_bool,
optional_str,
require_mapping,
require_str,
)
from entity.enum_options import enum_options_from_values
def _serialize_config(config: BaseConfig) -> Dict[str, Any]:
"""Serialize a config to dict, excluding the path field."""
payload: Dict[str, Any] = {}
for field_obj in fields(config):
if field_obj.name == "path":
continue
payload[field_obj.name] = getattr(config, field_obj.name)
return payload
class SplitTypeConfig(BaseConfig):
"""Base helper class for split type configs."""
def display_label(self) -> str:
return self.__class__.__name__
def to_external_value(self) -> Any:
return _serialize_config(self)
@dataclass
class MessageSplitConfig(SplitTypeConfig):
"""Configuration for message-based splitting.
Each input message becomes one execution unit. No additional configuration needed.
"""
FIELD_SPECS: ClassVar[Dict[str, ConfigFieldSpec]] = {}
@classmethod
def from_dict(cls, data: Mapping[str, Any] | None, *, path: str) -> "MessageSplitConfig":
# No config needed for message split
return cls(path=path)
def display_label(self) -> str:
return "message"
_NO_MATCH_DESCRIPTIONS = {
"pass": "Leave the content unchanged when no match is found.",
"empty": "Return empty content when no match is found.",
}
@dataclass
class RegexSplitConfig(SplitTypeConfig):
"""Configuration for regex-based splitting.
Split content by regex pattern matches. Each match becomes one execution unit.
Attributes:
pattern: Python regular expression used to split content.
group: Capture group name or index. Defaults to the entire match (group 0).
case_sensitive: Whether the regex should be case sensitive.
multiline: Enable multiline mode (re.MULTILINE).
dotall: Enable dotall mode (re.DOTALL).
on_no_match: Behavior when no match is found.
"""
pattern: str = ""
group: str | int | None = None
case_sensitive: bool = True
multiline: bool = False
dotall: bool = False
on_no_match: str = "pass"
FIELD_SPECS = {
"pattern": ConfigFieldSpec(
name="pattern",
display_name="Regex Pattern",
type_hint="str",
required=True,
description="Python regular expression used to split content.",
),
"group": ConfigFieldSpec(
name="group",
display_name="Capture Group",
type_hint="str",
required=False,
description="Capture group name or index. Defaults to the entire match (group 0).",
),
"case_sensitive": ConfigFieldSpec(
name="case_sensitive",
display_name="Case Sensitive",
type_hint="bool",
required=False,
default=True,
description="Whether the regex should be case sensitive.",
),
"multiline": ConfigFieldSpec(
name="multiline",
display_name="Multiline Flag",
type_hint="bool",
required=False,
default=False,
description="Enable multiline mode (re.MULTILINE).",
advance=True,
),
"dotall": ConfigFieldSpec(
name="dotall",
display_name="Dotall Flag",
type_hint="bool",
required=False,
default=False,
description="Enable dotall mode (re.DOTALL).",
advance=True,
),
"on_no_match": ConfigFieldSpec(
name="on_no_match",
display_name="No Match Behavior",
type_hint="enum",
required=False,
default="pass",
enum=["pass", "empty"],
description="Behavior when no match is found.",
enum_options=enum_options_from_values(
list(_NO_MATCH_DESCRIPTIONS.keys()),
_NO_MATCH_DESCRIPTIONS,
preserve_label_case=True,
),
advance=True,
),
}
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "RegexSplitConfig":
mapping = require_mapping(data, path)
pattern = require_str(mapping, "pattern", path, allow_empty=False)
group_value = mapping.get("group")
group_normalized: str | int | None = None
if group_value is not None:
if isinstance(group_value, int):
group_normalized = group_value
elif isinstance(group_value, str):
if group_value.isdigit():
group_normalized = int(group_value)
else:
group_normalized = group_value
else:
raise ConfigError("group must be str or int", extend_path(path, "group"))
case_sensitive = optional_bool(mapping, "case_sensitive", path, default=True)
multiline = optional_bool(mapping, "multiline", path, default=False)
dotall = optional_bool(mapping, "dotall", path, default=False)
on_no_match = optional_str(mapping, "on_no_match", path) or "pass"
if on_no_match not in {"pass", "empty"}:
raise ConfigError("on_no_match must be 'pass' or 'empty'", extend_path(path, "on_no_match"))
return cls(
pattern=pattern,
group=group_normalized,
case_sensitive=True if case_sensitive is None else bool(case_sensitive),
multiline=bool(multiline) if multiline is not None else False,
dotall=bool(dotall) if dotall is not None else False,
on_no_match=on_no_match,
path=path,
)
def display_label(self) -> str:
return f"regex({self.pattern})"
@dataclass
class JsonPathSplitConfig(SplitTypeConfig):
"""Configuration for JSON path-based splitting.
Split content by extracting array items from JSON using a path expression.
Each array item becomes one execution unit.
Attributes:
json_path: Simple dot-notation path to array (e.g., 'items', 'data.results').
"""
json_path: str = ""
FIELD_SPECS = {
"json_path": ConfigFieldSpec(
name="json_path",
display_name="JSON Path",
type_hint="str",
required=True,
description="Simple dot-notation path to array (e.g., 'items', 'data.results').",
),
}
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "JsonPathSplitConfig":
mapping = require_mapping(data, path)
json_path_value = require_str(mapping, "json_path", path, allow_empty=True)
return cls(json_path=json_path_value, path=path)
def display_label(self) -> str:
return f"json_path({self.json_path})"
# Registry for split types
_SPLIT_TYPE_REGISTRY: Dict[str, Dict[str, Any]] = {
"message": {
"config_cls": MessageSplitConfig,
"summary": "Each input message becomes one unit",
},
"regex": {
"config_cls": RegexSplitConfig,
"summary": "Split by regex pattern matches",
},
"json_path": {
"config_cls": JsonPathSplitConfig,
"summary": "Split by JSON array path",
},
}
def get_split_type_config(name: str) -> Type[SplitTypeConfig]:
"""Get the config class for a split type."""
entry = _SPLIT_TYPE_REGISTRY.get(name)
if not entry:
raise ConfigError(f"Unknown split type: {name}", None)
return entry["config_cls"]
def iter_split_type_registrations() -> Dict[str, Type[SplitTypeConfig]]:
"""Iterate over all registered split types."""
return {name: entry["config_cls"] for name, entry in _SPLIT_TYPE_REGISTRY.items()}
def iter_split_type_metadata() -> Dict[str, Dict[str, Any]]:
"""Iterate over split type metadata."""
return {name: {"summary": entry.get("summary")} for name, entry in _SPLIT_TYPE_REGISTRY.items()}
TSplitConfig = TypeVar("TSplitConfig", bound=SplitTypeConfig)
@dataclass
class SplitConfig(BaseConfig):
"""Configuration for how to split inputs into execution units.
Attributes:
type: Split strategy type (message, regex, json_path)
config: Type-specific configuration
"""
type: str = "message"
config: SplitTypeConfig | None = None
FIELD_SPECS = {
"type": ConfigFieldSpec(
name="type",
display_name="Split Type",
type_hint="str",
required=True,
default="message",
description="Strategy for splitting inputs into parallel execution units",
),
"config": ConfigFieldSpec(
name="config",
display_name="Split Config",
type_hint="object",
required=False,
description="Type-specific split configuration",
),
}
@classmethod
def child_routes(cls) -> Dict[ChildKey, Type[BaseConfig]]:
return {
ChildKey(field="config", value=name): config_cls
for name, config_cls in iter_split_type_registrations().items()
}
@classmethod
def field_specs(cls) -> Dict[str, ConfigFieldSpec]:
specs = super().field_specs()
type_spec = specs.get("type")
if type_spec:
registrations = iter_split_type_registrations()
metadata = iter_split_type_metadata()
type_names = list(registrations.keys())
descriptions = {name: (metadata.get(name) or {}).get("summary") for name in type_names}
specs["type"] = replace(
type_spec,
enum=type_names,
enum_options=enum_options_from_values(type_names, descriptions),
)
return specs
@classmethod
def from_dict(cls, data: Mapping[str, Any] | None, *, path: str) -> "SplitConfig":
if data is None:
# Default to message split
return cls(type="message", config=MessageSplitConfig(path=extend_path(path, "config")), path=path)
mapping = require_mapping(data, path)
split_type = optional_str(mapping, "type", path) or "message"
if split_type not in _SPLIT_TYPE_REGISTRY:
raise ConfigError(
f"split type must be one of {list(_SPLIT_TYPE_REGISTRY.keys())}, got '{split_type}'",
extend_path(path, "type"),
)
config_cls = get_split_type_config(split_type)
config_data = mapping.get("config")
config_path = extend_path(path, "config")
# For message type, config is optional
if split_type == "message":
config = config_cls.from_dict(config_data, path=config_path)
else:
if config_data is None:
raise ConfigError(f"{split_type} split requires 'config' field", path)
config = config_cls.from_dict(config_data, path=config_path)
return cls(type=split_type, config=config, path=path)
def display_label(self) -> str:
if self.config:
return self.config.display_label()
return self.type
def to_external_value(self) -> Any:
return {
"type": self.type,
"config": self.config.to_external_value() if self.config else {},
}
def as_split_config(self, expected_type: Type[TSplitConfig]) -> TSplitConfig | None:
"""Return the nested config if it matches the expected type."""
if isinstance(self.config, expected_type):
return self.config
return None
# Convenience properties for backward compatibility and easy access
@property
def pattern(self) -> Optional[str]:
"""Get regex pattern if this is a regex split."""
if isinstance(self.config, RegexSplitConfig):
return self.config.pattern
return None
@property
def json_path(self) -> Optional[str]:
"""Get json_path if this is a json_path split."""
if isinstance(self.config, JsonPathSplitConfig):
return self.config.json_path
return None
@dataclass
class MapDynamicConfig(BaseConfig):
"""Configuration for Map dynamic mode (fan-out only).
Map mode is similar to passthrough - minimal config required.
Attributes:
max_parallel: Maximum concurrent executions
"""
max_parallel: int = 10
FIELD_SPECS = {
"max_parallel": ConfigFieldSpec(
name="max_parallel",
display_name="Max Parallel",
type_hint="int",
required=False,
default=10,
description="Maximum number of parallel executions",
),
}
@classmethod
def from_dict(cls, data: Mapping[str, Any] | None, *, path: str) -> "MapDynamicConfig":
if data is None:
return cls(path=path)
mapping = require_mapping(data, path)
max_parallel = int(mapping.get("max_parallel", 10))
return cls(max_parallel=max_parallel, path=path)
@dataclass
class TreeDynamicConfig(BaseConfig):
"""Configuration for Tree dynamic mode (fan-out and reduce).
Attributes:
group_size: Number of items per group in reduction
max_parallel: Maximum concurrent executions per layer
"""
group_size: int = 3
max_parallel: int = 10
FIELD_SPECS = {
"group_size": ConfigFieldSpec(
name="group_size",
display_name="Group Size",
type_hint="int",
required=False,
default=3,
description="Number of items per group during reduction",
),
"max_parallel": ConfigFieldSpec(
name="max_parallel",
display_name="Max Parallel",
type_hint="int",
required=False,
default=10,
description="Maximum concurrent executions per layer",
),
}
@classmethod
def from_dict(cls, data: Mapping[str, Any] | None, *, path: str) -> "TreeDynamicConfig":
if data is None:
return cls(path=path)
mapping = require_mapping(data, path)
group_size = int(mapping.get("group_size", 3))
if group_size < 2:
raise ConfigError("group_size must be at least 2", extend_path(path, "group_size"))
max_parallel = int(mapping.get("max_parallel", 10))
return cls(group_size=group_size, max_parallel=max_parallel, path=path)

17
entity/configs/edge/__init__.py Executable file
View File

@ -0,0 +1,17 @@
from .edge import EdgeConfig
from .edge_condition import EdgeConditionConfig
from .edge_processor import (
EdgeProcessorConfig,
RegexEdgeProcessorConfig,
FunctionEdgeProcessorConfig,
)
from .dynamic_edge_config import DynamicEdgeConfig
__all__ = [
"EdgeConfig",
"EdgeConditionConfig",
"EdgeProcessorConfig",
"RegexEdgeProcessorConfig",
"FunctionEdgeProcessorConfig",
"DynamicEdgeConfig",
]

183
entity/configs/edge/dynamic_edge_config.py Executable file
View File

@ -0,0 +1,183 @@
"""Dynamic edge configuration for edge-level Map and Tree execution modes."""
from dataclasses import dataclass, field, replace
from typing import Any, Dict, Mapping
from entity.configs.base import (
BaseConfig,
ConfigError,
ConfigFieldSpec,
ChildKey,
extend_path,
require_mapping,
require_str,
)
from entity.configs.dynamic_base import (
SplitConfig,
MapDynamicConfig,
TreeDynamicConfig,
)
from entity.enum_options import enum_options_from_values
from utils.registry import Registry, RegistryError
# Local registry for edge-level dynamic types (reuses same type names)
dynamic_edge_type_registry = Registry("dynamic_edge_type")
def register_dynamic_edge_type(
name: str,
*,
config_cls: type[BaseConfig],
description: str | None = None,
) -> None:
metadata = {"summary": description} if description else None
dynamic_edge_type_registry.register(name, target=config_cls, metadata=metadata)
def get_dynamic_edge_type_config(name: str) -> type[BaseConfig]:
entry = dynamic_edge_type_registry.get(name)
config_cls = entry.load()
if not isinstance(config_cls, type) or not issubclass(config_cls, BaseConfig):
raise RegistryError(f"Entry '{name}' is not a BaseConfig subclass")
return config_cls
def iter_dynamic_edge_type_registrations() -> Dict[str, type[BaseConfig]]:
return {name: entry.load() for name, entry in dynamic_edge_type_registry.items()}
def iter_dynamic_edge_type_metadata() -> Dict[str, Dict[str, Any]]:
return {name: dict(entry.metadata or {}) for name, entry in dynamic_edge_type_registry.items()}
@dataclass
class DynamicEdgeConfig(BaseConfig):
"""Dynamic configuration for edge-level Map and Tree execution modes.
When configured on an edge, the target node will be dynamically expanded
based on the split results. The split logic is applied to messages
passing through this edge.
Attributes:
type: Dynamic mode type (map or tree)
split: How to split the payload passing through this edge
config: Mode-specific configuration (MapDynamicConfig or TreeDynamicConfig)
"""
type: str
split: SplitConfig = field(default_factory=lambda: SplitConfig())
config: BaseConfig | None = None
FIELD_SPECS = {
"type": ConfigFieldSpec(
name="type",
display_name="Dynamic Type",
type_hint="str",
required=True,
description="Dynamic execution mode (map or tree)",
),
"split": ConfigFieldSpec(
name="split",
display_name="Split Strategy",
type_hint="SplitConfig",
required=False,
description="How to split the edge payload into parallel execution units",
child=SplitConfig,
),
"config": ConfigFieldSpec(
name="config",
display_name="Dynamic Config",
type_hint="object",
required=False,
description="Mode-specific configuration",
),
}
@classmethod
def child_routes(cls) -> Dict[ChildKey, type[BaseConfig]]:
return {
ChildKey(field="config", value=name): config_cls
for name, config_cls in iter_dynamic_edge_type_registrations().items()
}
@classmethod
def field_specs(cls) -> Dict[str, ConfigFieldSpec]:
specs = super().field_specs()
type_spec = specs.get("type")
if type_spec:
registrations = iter_dynamic_edge_type_registrations()
metadata = iter_dynamic_edge_type_metadata()
type_names = list(registrations.keys())
descriptions = {name: (metadata.get(name) or {}).get("summary") for name in type_names}
specs["type"] = replace(
type_spec,
enum=type_names,
enum_options=enum_options_from_values(type_names, descriptions),
)
return specs
@classmethod
def from_dict(cls, data: Mapping[str, Any] | None, *, path: str) -> "DynamicEdgeConfig | None":
if data is None:
return None
mapping = require_mapping(data, path)
dynamic_type = require_str(mapping, "type", path)
try:
config_cls = get_dynamic_edge_type_config(dynamic_type)
except RegistryError as exc:
raise ConfigError(
f"dynamic type must be one of {list(iter_dynamic_edge_type_registrations().keys())}",
extend_path(path, "type"),
) from exc
# Parse split at top level
split_data = mapping.get("split")
split = SplitConfig.from_dict(split_data, path=extend_path(path, "split"))
# Parse mode-specific config
config_data = mapping.get("config")
config_path = extend_path(path, "config")
config = config_cls.from_dict(config_data, path=config_path)
return cls(type=dynamic_type, split=split, config=config, path=path)
def is_map(self) -> bool:
return self.type == "map"
def is_tree(self) -> bool:
return self.type == "tree"
def as_map_config(self) -> MapDynamicConfig | None:
return self.config if self.is_map() and isinstance(self.config, MapDynamicConfig) else None
def as_tree_config(self) -> TreeDynamicConfig | None:
return self.config if self.is_tree() and isinstance(self.config, TreeDynamicConfig) else None
@property
def max_parallel(self) -> int:
"""Get max_parallel from config."""
if hasattr(self.config, "max_parallel"):
return getattr(self.config, "max_parallel")
return 10
@property
def group_size(self) -> int:
"""Get group_size (tree mode only, defaults to 3)."""
if isinstance(self.config, TreeDynamicConfig):
return self.config.group_size
return 3
# Register dynamic edge types
register_dynamic_edge_type(
"map",
config_cls=MapDynamicConfig,
description="Fan-out only: split into parallel units and collect results",
)
register_dynamic_edge_type(
"tree",
config_cls=TreeDynamicConfig,
description="Fan-out and reduce: split into units, then iteratively reduce results",
)

151
entity/configs/edge/edge.py Executable file
View File

@ -0,0 +1,151 @@
"""Edge configuration dataclasses."""
from dataclasses import dataclass, field
from typing import Any, Dict, Mapping
from entity.configs.base import (
BaseConfig,
ConfigFieldSpec,
require_mapping,
require_str,
optional_bool,
extend_path,
)
from .edge_condition import EdgeConditionConfig
from .edge_processor import EdgeProcessorConfig
from .dynamic_edge_config import DynamicEdgeConfig
@dataclass
class EdgeConfig(BaseConfig):
source: str
target: str
trigger: bool = True
condition: EdgeConditionConfig | None = None
carry_data: bool = True
keep_message: bool = False
clear_context: bool = False
clear_kept_context: bool = False
process: EdgeProcessorConfig | None = None
dynamic: DynamicEdgeConfig | None = None
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "EdgeConfig":
mapping = require_mapping(data, path)
source = require_str(mapping, "from", path)
target = require_str(mapping, "to", path)
trigger_value = optional_bool(mapping, "trigger", path, default=True)
carry_data_value = optional_bool(mapping, "carry_data", path, default=True)
keep_message_value = optional_bool(mapping, "keep_message", path, default=False)
clear_context_value = optional_bool(mapping, "clear_context", path, default=False)
clear_kept_context_value = optional_bool(mapping, "clear_kept_context", path, default=False)
condition_value = mapping.get("condition", "true")
condition_cfg = EdgeConditionConfig.from_dict(condition_value, path=extend_path(path, "condition"))
process_cfg = None
if "process" in mapping and mapping["process"] is not None:
process_cfg = EdgeProcessorConfig.from_dict(mapping["process"], path=extend_path(path, "process"))
dynamic_cfg = None
if "dynamic" in mapping and mapping["dynamic"] is not None:
dynamic_cfg = DynamicEdgeConfig.from_dict(mapping["dynamic"], path=extend_path(path, "dynamic"))
return cls(
source=source,
target=target,
trigger=bool(trigger_value) if trigger_value is not None else True,
condition=condition_cfg,
carry_data=bool(carry_data_value) if carry_data_value is not None else True,
keep_message=bool(keep_message_value) if keep_message_value is not None else False,
clear_context=bool(clear_context_value) if clear_context_value is not None else False,
clear_kept_context=bool(clear_kept_context_value) if clear_kept_context_value is not None else False,
process=process_cfg,
dynamic=dynamic_cfg,
path=path,
)
FIELD_SPECS = {
"from": ConfigFieldSpec(
name="from",
display_name="Source Node ID",
type_hint="str",
required=True,
description="Source node ID of the edge",
),
"to": ConfigFieldSpec(
name="to",
display_name="Target Node ID",
type_hint="str",
required=True,
description="Target node ID of the edge",
),
"trigger": ConfigFieldSpec(
name="trigger",
type_hint="bool",
required=False,
default=True,
display_name="Can Trigger Successor",
description="Whether this edge can trigger successor nodes",
advance=True,
),
"condition": ConfigFieldSpec(
name="condition",
type_hint="EdgeConditionConfig",
required=False,
display_name="Edge Condition",
description="Edge condition configuration（type + config）",
advance=True,
child=EdgeConditionConfig,
),
"carry_data": ConfigFieldSpec(
name="carry_data",
type_hint="bool",
required=False,
default=True,
display_name="Pass Data to Target",
description="Whether to pass data to the target node",
advance=True,
),
"keep_message": ConfigFieldSpec(
name="keep_message",
type_hint="bool",
required=False,
default=False,
display_name="Keep Message Input",
description="Whether to always keep this message input in the target node without being cleared",
advance=True,
),
"clear_context": ConfigFieldSpec(
name="clear_context",
type_hint="bool",
required=False,
default=False,
display_name="Clear Context",
description="Clear all incoming context messages without keep=True before passing new payload",
advance=True,
),
"clear_kept_context": ConfigFieldSpec(
name="clear_kept_context",
type_hint="bool",
required=False,
default=False,
display_name="Clear Kept Context",
description="Clear messages marked with keep=True before passing new payload",
advance=True,
),
"process": ConfigFieldSpec(
name="process",
type_hint="EdgeProcessorConfig",
required=False,
display_name="Payload Processor",
description="Optional payload processor applied after the condition is met (regex extraction, custom functions, etc.)",
advance=True,
child=EdgeProcessorConfig,
),
"dynamic": ConfigFieldSpec(
name="dynamic",
type_hint="DynamicEdgeConfig",
required=False,
display_name="Dynamic Expansion",
description="Dynamic expansion configuration for edge-level Map (fan-out) or Tree (fan-out + reduce) modes. When set, the target node is dynamically expanded based on split results.",
advance=True,
child=DynamicEdgeConfig,
),
}

302
entity/configs/edge/edge_condition.py Executable file
View File

@ -0,0 +1,302 @@
"""Edge condition configuration models."""
from dataclasses import dataclass, field, fields, replace
from typing import Any, Dict, Mapping, Type, TypeVar, cast
from entity.enum_options import enum_options_from_values
from schema_registry import (
SchemaLookupError,
get_edge_condition_schema,
iter_edge_condition_schemas,
)
from entity.configs.base import (
BaseConfig,
ChildKey,
ConfigError,
ConfigFieldSpec,
ensure_list,
optional_bool,
require_mapping,
require_str,
extend_path,
)
from utils.function_catalog import get_function_catalog
from utils.function_manager import EDGE_FUNCTION_DIR
def _serialize_config(config: BaseConfig) -> Dict[str, Any]:
payload: Dict[str, Any] = {}
for field_obj in fields(config):
if field_obj.name == "path":
continue
payload[field_obj.name] = getattr(config, field_obj.name)
return payload
class EdgeConditionTypeConfig(BaseConfig):
"""Base helper for condition-specific configuration classes."""
def display_label(self) -> str:
return self.__class__.__name__
def to_external_value(self) -> Any:
return _serialize_config(self)
@dataclass
class FunctionEdgeConditionConfig(EdgeConditionTypeConfig):
"""Configuration for function-based conditions."""
name: str = "true"
FIELD_SPECS = {
"name": ConfigFieldSpec(
name="name",
display_name="Function Name",
type_hint="str",
required=True,
default="true",
description="Function Name or 'true' (indicating perpetual satisfaction)",
)
}
@classmethod
def from_dict(cls, data: Mapping[str, Any] | None, *, path: str) -> "FunctionEdgeConditionConfig":
if data is None:
return cls(name="true", path=path)
mapping = require_mapping(data, path)
function_name = require_str(mapping, "name", path, allow_empty=False)
return cls(name=function_name, path=path)
@classmethod
def field_specs(cls) -> Dict[str, ConfigFieldSpec]:
specs = super().field_specs()
name_spec = specs.get("name")
if name_spec is None:
return specs
catalog = get_function_catalog(EDGE_FUNCTION_DIR)
names = catalog.list_function_names()
metadata = catalog.list_metadata()
description = name_spec.description or "Conditional function name"
if catalog.load_error:
description = f"{description} (Loading failed: {catalog.load_error})"
elif not names:
description = f"{description} (No available conditional functions found)"
if "true" not in names:
names.insert(0, "true")
descriptions = {"true": "Default condition (always met)"}
for name in names:
if name == "true":
continue
meta = metadata.get(name)
descriptions[name] = (meta.description if meta else None) or "The conditional function is not described."
specs["name"] = replace(
name_spec,
enum=names or None,
enum_options=enum_options_from_values(names, descriptions, preserve_label_case=True),
description=description,
)
return specs
def display_label(self) -> str:
return self.name or "true"
def to_external_value(self) -> Any:
return self.name or "true"
def _normalize_keyword_list(value: Any, path: str) -> list[str]:
items = ensure_list(value)
normalized: list[str] = []
for idx, item in enumerate(items):
if not isinstance(item, str):
raise ConfigError("entries must be strings", extend_path(path, f"[{idx}]"))
normalized.append(item)
return normalized
@dataclass
class KeywordEdgeConditionConfig(EdgeConditionTypeConfig):
"""Configuration for declarative keyword checks."""
any_keywords: list[str] = field(default_factory=list)
none_keywords: list[str] = field(default_factory=list)
regex_patterns: list[str] = field(default_factory=list)
case_sensitive: bool = True
default: bool = False
FIELD_SPECS = {
"any": ConfigFieldSpec(
name="any",
display_name="Contains keywords",
type_hint="list[str]",
required=False,
description="Returns True if any keyword is matched.",
),
"none": ConfigFieldSpec(
name="none",
display_name="Exclude keywords",
type_hint="list[str]",
required=False,
description="If any of the excluded keywords are matched, return False (highest priority).",
),
"regex": ConfigFieldSpec(
name="regex",
display_name="Regular expressions",
type_hint="list[str]",
required=False,
description="Returns True if any regular expression is matched.",
advance=True,
),
"case_sensitive": ConfigFieldSpec(
name="case_sensitive",
display_name="case sensitive",
type_hint="bool",
required=False,
default=True,
description="Whether to distinguish between uppercase and lowercase letters (default is true).",
),
# "default": ConfigFieldSpec(
# name="default",
# display_name="Default Result",
# type_hint="bool",
# required=False,
# default=False,
# description="Return value when no condition matches; defaults to False",
# advance=True,
# ),
}
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "KeywordEdgeConditionConfig":
mapping = require_mapping(data, path)
any_keywords = _normalize_keyword_list(mapping.get("any", []), extend_path(path, "any"))
none_keywords = _normalize_keyword_list(mapping.get("none", []), extend_path(path, "none"))
regex_patterns = _normalize_keyword_list(mapping.get("regex", []), extend_path(path, "regex"))
case_sensitive = optional_bool(mapping, "case_sensitive", path, default=True)
default_value = optional_bool(mapping, "default", path, default=False)
if not (any_keywords or none_keywords or regex_patterns):
raise ConfigError("keyword condition requires any/none/regex", path)
return cls(
any_keywords=any_keywords,
none_keywords=none_keywords,
regex_patterns=regex_patterns,
case_sensitive=True if case_sensitive is None else bool(case_sensitive),
default=False if default_value is None else bool(default_value),
path=path,
)
def display_label(self) -> str:
return f"keyword(any={len(self.any_keywords)}, none={len(self.none_keywords)}, regex={len(self.regex_patterns)})"
def to_external_value(self) -> Any:
payload: Dict[str, Any] = {}
if self.any_keywords:
payload["any"] = list(self.any_keywords)
if self.none_keywords:
payload["none"] = list(self.none_keywords)
if self.regex_patterns:
payload["regex"] = list(self.regex_patterns)
payload["case_sensitive"] = self.case_sensitive
payload["default"] = self.default
return payload
TConditionConfig = TypeVar("TConditionConfig", bound=EdgeConditionTypeConfig)
@dataclass
class EdgeConditionConfig(BaseConfig):
"""Wrapper config that stores condition type + concrete config."""
type: str
config: EdgeConditionTypeConfig
FIELD_SPECS = {
"type": ConfigFieldSpec(
name="type",
display_name="Condition Type",
type_hint="str",
required=True,
description="Select which condition implementation to run (function, keyword, etc.) so the engine can resolve the schema.",
),
"config": ConfigFieldSpec(
name="config",
display_name="Condition Config",
type_hint="object",
required=True,
description="Payload interpreted by the chosen function or any/none/regex lists for keyword mode.",
),
}
@classmethod
def _normalize_value(cls, value: Any, path: str) -> Mapping[str, Any]:
if value is None:
return {"type": "function", "config": {"name": "true"}}
if isinstance(value, bool):
if value:
return {"type": "function", "config": {"name": "true"}}
return {"type": "function", "config": {"name": "always_false"}}
if isinstance(value, str):
return {"type": "function", "config": {"name": value}}
return require_mapping(value, path)
@classmethod
def from_dict(cls, data: Any, *, path: str) -> "EdgeConditionConfig":
mapping = cls._normalize_value(data, path)
condition_type = require_str(mapping, "type", path)
config_payload = mapping.get("config")
config_path = extend_path(path, "config")
try:
schema = get_edge_condition_schema(condition_type)
except SchemaLookupError as exc:
raise ConfigError(f"unknown condition type '{condition_type}'", extend_path(path, "type")) from exc
if config_payload is None:
raise ConfigError("condition config is required", config_path)
condition_config = schema.config_cls.from_dict(config_payload, path=config_path)
return cls(type=condition_type, config=condition_config, path=path)
@classmethod
def child_routes(cls) -> Dict[ChildKey, Type[BaseConfig]]:
return {
ChildKey(field="config", value=name): schema.config_cls
for name, schema in iter_edge_condition_schemas().items()
}
@classmethod
def field_specs(cls) -> Dict[str, ConfigFieldSpec]:
specs = super().field_specs()
type_spec = specs.get("type")
if type_spec:
registrations = iter_edge_condition_schemas()
names = list(registrations.keys())
descriptions = {name: schema.summary for name, schema in registrations.items()}
specs["type"] = replace(
type_spec,
enum=names,
enum_options=enum_options_from_values(names, descriptions, preserve_label_case=True),
)
return specs
def display_label(self) -> str:
return self.config.display_label()
def to_external_value(self) -> Any:
if self.type == "function":
return self.config.to_external_value()
return {
"type": self.type,
"config": self.config.to_external_value(),
}
def as_config(self, expected_type: Type[TConditionConfig]) -> TConditionConfig | None:
config = self.config
if isinstance(config, expected_type):
return cast(TConditionConfig, config)
return None

334
entity/configs/edge/edge_processor.py Executable file
View File

@ -0,0 +1,334 @@
"""Edge payload processor configuration dataclasses."""
from dataclasses import dataclass, field, fields, replace
from typing import Any, Dict, Mapping, Type, TypeVar, cast
from entity.enum_options import enum_options_from_values
from utils.function_catalog import get_function_catalog
from utils.function_manager import EDGE_PROCESSOR_FUNCTION_DIR
from schema_registry import (
SchemaLookupError,
get_edge_processor_schema,
iter_edge_processor_schemas,
)
from entity.configs.base import (
BaseConfig,
ChildKey,
ConfigError,
ConfigFieldSpec,
ensure_list,
optional_bool,
optional_str,
require_mapping,
require_str,
extend_path,
)
def _serialize_config(config: BaseConfig) -> Dict[str, Any]:
payload: Dict[str, Any] = {}
for field_obj in fields(config):
if field_obj.name == "path":
continue
payload[field_obj.name] = getattr(config, field_obj.name)
return payload
class EdgeProcessorTypeConfig(BaseConfig):
"""Base helper class for payload processor configs."""
def display_label(self) -> str:
return self.__class__.__name__
def to_external_value(self) -> Any:
return _serialize_config(self)
_NO_MATCH_DESCRIPTIONS = {
"pass": "Leave the payload untouched when no match is found.",
"default": "Apply default_value (or empty string) if nothing matches.",
"drop": "Discard the payload entirely when the regex does not match.",
}
@dataclass
class RegexEdgeProcessorConfig(EdgeProcessorTypeConfig):
"""Configuration for regex-based payload extraction."""
pattern: str = ""
group: str | int | None = None
case_sensitive: bool = True
multiline: bool = False
dotall: bool = False
multiple: bool = False
template: str | None = None
on_no_match: str = "pass"
default_value: str | None = None
FIELD_SPECS = {
"pattern": ConfigFieldSpec(
name="pattern",
display_name="Regex Pattern",
type_hint="str",
required=True,
description="Python regular expression used to extract content.",
),
"group": ConfigFieldSpec(
name="group",
display_name="Capture Group",
type_hint="str",
required=False,
description="Capture group name or index. Defaults to the entire match.",
),
"case_sensitive": ConfigFieldSpec(
name="case_sensitive",
display_name="Case Sensitive",
type_hint="bool",
required=False,
default=True,
description="Whether the regex should be case sensitive.",
),
"multiline": ConfigFieldSpec(
name="multiline",
display_name="Multiline Flag",
type_hint="bool",
required=False,
default=False,
description="Enable multiline mode (re.MULTILINE).",
advance=True,
),
"dotall": ConfigFieldSpec(
name="dotall",
display_name="Dotall Flag",
type_hint="bool",
required=False,
default=False,
description="Enable dotall mode (re.DOTALL).",
advance=True,
),
"multiple": ConfigFieldSpec(
name="multiple",
display_name="Return Multiple Matches",
type_hint="bool",
required=False,
default=False,
description="Whether to collect all matches instead of only the first.",
advance=True,
),
"template": ConfigFieldSpec(
name="template",
display_name="Output Template",
type_hint="str",
required=False,
description="Optional template applied to the extracted value. Use '{match}' placeholder.",
advance=True,
),
"on_no_match": ConfigFieldSpec(
name="on_no_match",
display_name="No Match Behavior",
type_hint="enum",
required=False,
default="pass",
enum=["pass", "default", "drop"],
description="Behavior when no match is found.",
enum_options=enum_options_from_values(
list(_NO_MATCH_DESCRIPTIONS.keys()),
_NO_MATCH_DESCRIPTIONS,
preserve_label_case=True,
),
advance=True,
),
"default_value": ConfigFieldSpec(
name="default_value",
display_name="Default Value",
type_hint="str",
required=False,
description="Fallback content when on_no_match=default.",
advance=True,
),
}
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "RegexEdgeProcessorConfig":
mapping = require_mapping(data, path)
pattern = require_str(mapping, "pattern", path, allow_empty=False)
group_value = mapping.get("group")
group_normalized: str | int | None = None
if group_value is not None:
if isinstance(group_value, int):
group_normalized = group_value
elif isinstance(group_value, str):
if group_value.isdigit():
group_normalized = int(group_value)
else:
group_normalized = group_value
else:
raise ConfigError("group must be str or int", extend_path(path, "group"))
multiple = optional_bool(mapping, "multiple", path, default=False)
case_sensitive = optional_bool(mapping, "case_sensitive", path, default=True)
multiline = optional_bool(mapping, "multiline", path, default=False)
dotall = optional_bool(mapping, "dotall", path, default=False)
on_no_match = optional_str(mapping, "on_no_match", path) or "pass"
if on_no_match not in {"pass", "default", "drop"}:
raise ConfigError("on_no_match must be pass, default or drop", extend_path(path, "on_no_match"))
template = optional_str(mapping, "template", path)
default_value = optional_str(mapping, "default_value", path)
return cls(
pattern=pattern,
group=group_normalized,
case_sensitive=True if case_sensitive is None else bool(case_sensitive),
multiline=bool(multiline) if multiline is not None else False,
dotall=bool(dotall) if dotall is not None else False,
multiple=bool(multiple) if multiple is not None else False,
template=template,
on_no_match=on_no_match,
default_value=default_value,
path=path,
)
def display_label(self) -> str:
return f"regex({self.pattern})"
@dataclass
class FunctionEdgeProcessorConfig(EdgeProcessorTypeConfig):
"""Configuration for function-based payload processors."""
name: str = ""
FIELD_SPECS = {
"name": ConfigFieldSpec(
name="name",
display_name="Function Name",
type_hint="str",
required=True,
description="Name of the Python function located in functions/edge_processor.",
)
}
@classmethod
def field_specs(cls) -> Dict[str, ConfigFieldSpec]:
specs = super().field_specs()
name_spec = specs.get("name")
if not name_spec:
return specs
catalog = get_function_catalog(EDGE_PROCESSOR_FUNCTION_DIR)
names = catalog.list_function_names()
metadata = catalog.list_metadata()
description = name_spec.description or "Processor function name"
if catalog.load_error:
description = f"{description} (Loading failed: {catalog.load_error})"
elif not names:
description = f"{description} (No processor functions found in functions/edge_processor)"
descriptions = {}
for name in names:
meta = metadata.get(name)
descriptions[name] = (meta.description if meta else None) or "No description provided."
specs["name"] = replace(
name_spec,
enum=names or None,
enum_options=enum_options_from_values(names, descriptions, preserve_label_case=True) if names else None,
description=description,
)
return specs
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "FunctionEdgeProcessorConfig":
mapping = require_mapping(data, path)
name = require_str(mapping, "name", path, allow_empty=False)
return cls(name=name, path=path)
def display_label(self) -> str:
return self.name or "function"
def to_external_value(self) -> Any:
return {"name": self.name}
TProcessorConfig = TypeVar("TProcessorConfig", bound=EdgeProcessorTypeConfig)
@dataclass
class EdgeProcessorConfig(BaseConfig):
"""Wrapper config storing processor type and payload."""
type: str
config: EdgeProcessorTypeConfig
FIELD_SPECS = {
"type": ConfigFieldSpec(
name="type",
display_name="Processor Type",
type_hint="str",
required=True,
description="Select which processor implementation to use (regex_extract, function, etc.).",
),
"config": ConfigFieldSpec(
name="config",
display_name="Processor Config",
type_hint="object",
required=True,
description="Payload interpreted by the selected processor.",
),
}
@classmethod
def from_dict(cls, data: Any, *, path: str) -> "EdgeProcessorConfig":
if data is None:
raise ConfigError("processor configuration cannot be null", path)
mapping = require_mapping(data, path)
processor_type = require_str(mapping, "type", path)
config_payload = mapping.get("config")
if config_payload is None:
raise ConfigError("processor config is required", extend_path(path, "config"))
try:
schema = get_edge_processor_schema(processor_type)
except SchemaLookupError as exc:
raise ConfigError(f"unknown processor type '{processor_type}'", extend_path(path, "type")) from exc
processor_config = schema.config_cls.from_dict(config_payload, path=extend_path(path, "config"))
return cls(type=processor_type, config=processor_config, path=path)
@classmethod
def child_routes(cls) -> Dict[ChildKey, Type[BaseConfig]]:
return {
ChildKey(field="config", value=name): schema.config_cls
for name, schema in iter_edge_processor_schemas().items()
}
@classmethod
def field_specs(cls) -> Dict[str, ConfigFieldSpec]:
specs = super().field_specs()
type_spec = specs.get("type")
if type_spec:
registrations = iter_edge_processor_schemas()
names = list(registrations.keys())
descriptions = {name: schema.summary for name, schema in registrations.items()}
specs["type"] = replace(
type_spec,
enum=names,
enum_options=enum_options_from_values(names, descriptions, preserve_label_case=True),
)
return specs
def display_label(self) -> str:
return self.config.display_label()
def to_external_value(self) -> Any:
return {
"type": self.type,
"config": self.config.to_external_value(),
}
def as_config(self, expected_type: Type[TProcessorConfig]) -> TProcessorConfig | None:
config = self.config
if isinstance(config, expected_type):
return cast(TProcessorConfig, config)
return None

313
entity/configs/graph.py Executable file
View File

@ -0,0 +1,313 @@
"""Graph-level configuration dataclasses."""
from dataclasses import dataclass, field
from collections import Counter
from typing import Any, Dict, List, Mapping
from entity.enums import LogLevel
from entity.enum_options import enum_options_for
from .base import (
BaseConfig,
ConfigError,
ConfigFieldSpec,
ensure_list,
optional_bool,
optional_dict,
optional_str,
require_mapping,
extend_path,
)
from .edge import EdgeConfig
from entity.configs.node.memory import MemoryStoreConfig
from entity.configs.node.agent import AgentConfig
from entity.configs.node.node import Node
@dataclass
class GraphDefinition(BaseConfig):
id: str | None
description: str | None
log_level: LogLevel
is_majority_voting: bool
nodes: List[Node] = field(default_factory=list)
edges: List[EdgeConfig] = field(default_factory=list)
memory: List[MemoryStoreConfig] | None = None
organization: str | None = None
initial_instruction: str | None = None
start_nodes: List[str] = field(default_factory=list)
end_nodes: List[str] | None = None
FIELD_SPECS = {
"id": ConfigFieldSpec(
name="id",
display_name="Graph ID",
type_hint="str",
required=True,
description="Graph identifier for referencing. Can only contain alphanumeric characters, underscores or hyphens, no spaces",
),
"description": ConfigFieldSpec(
name="description",
display_name="Graph Description",
type_hint="text",
required=False,
description="Human-readable narrative shown in UI/templates that explains the workflow goal, scope, and manual touchpoints.",
),
"log_level": ConfigFieldSpec(
name="log_level",
display_name="Log Level",
type_hint="enum:LogLevel",
required=False,
default=LogLevel.DEBUG.value,
enum=[lvl.value for lvl in LogLevel],
description="Runtime log level",
advance=True,
enum_options=enum_options_for(LogLevel),
),
"is_majority_voting": ConfigFieldSpec(
name="is_majority_voting",
display_name="Majority Voting Mode",
type_hint="bool",
required=False,
default=False,
description="Whether this is a majority voting graph",
advance=True,
),
"nodes": ConfigFieldSpec(
name="nodes",
display_name="Node List",
type_hint="list[Node]",
required=False,
description="Node list, must contain at least one node",
child=Node,
),
"edges": ConfigFieldSpec(
name="edges",
display_name="Edge List",
type_hint="list[EdgeConfig]",
required=False,
description="Directed edges between nodes",
child=EdgeConfig,
),
"memory": ConfigFieldSpec(
name="memory",
display_name="Memory Stores",
type_hint="list[MemoryStoreConfig]",
required=False,
description="Optional list of memory stores that nodes can reference through their model.memories attachments.",
child=MemoryStoreConfig,
),
# "organization": ConfigFieldSpec(
# name="organization",
# display_name="Organization Name",
# type_hint="str",
# required=False,
# description="Organization name",
# ),
"initial_instruction": ConfigFieldSpec(
name="initial_instruction",
display_name="Initial Instruction",
type_hint="text",
required=False,
description="Graph level initial instruction (for user)",
),
"start": ConfigFieldSpec(
name="start",
display_name="Start Node",
type_hint="list[str]",
required=False,
description="Start node ID list (entry list executed at workflow start; not recommended to edit manually)",
advance=True,
),
"end": ConfigFieldSpec(
name="end",
display_name="End Node",
type_hint="list[str]",
required=False,
description="End node ID list (used to collect final graph output, not part of execution logic). Commonly needed in subgraphs. This is an ordered list: earlier nodes are checked first; the first with output becomes the graph output, otherwise continue down the list.",
advance=True,
),
}
# CONSTRAINTS = (
# RuntimeConstraint(
# when={"memory": "*"},
# require=["memory"],
# message="After defining memory, at least one store must be declared",
# ),
# )
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "GraphDefinition":
mapping = require_mapping(data, path)
graph_id = optional_str(mapping, "id", path)
description = optional_str(mapping, "description", path)
if "vars" in mapping and mapping["vars"]:
raise ConfigError("vars are only supported at DesignConfig root", extend_path(path, "vars"))
log_level_raw = mapping.get("log_level", LogLevel.DEBUG.value)
try:
log_level = LogLevel(log_level_raw)
except ValueError as exc:
raise ConfigError(
f"log_level must be one of {[lvl.value for lvl in LogLevel]}", extend_path(path, "log_level")
) from exc
is_majority = optional_bool(mapping, "is_majority_voting", path, default=False)
organization = optional_str(mapping, "organization", path)
initial_instruction = optional_str(mapping, "initial_instruction", path)
nodes_raw = ensure_list(mapping.get("nodes"))
# if not nodes_raw:
# raise ConfigError("graph must define at least one node", extend_path(path, "nodes"))
nodes: List[Node] = []
for idx, node_dict in enumerate(nodes_raw):
nodes.append(Node.from_dict(node_dict, path=extend_path(path, f"nodes[{idx}]")))
edges_raw = ensure_list(mapping.get("edges"))
edges: List[EdgeConfig] = []
for idx, edge_dict in enumerate(edges_raw):
edges.append(EdgeConfig.from_dict(edge_dict, path=extend_path(path, f"edges[{idx}]")))
memory_cfg: List[MemoryStoreConfig] | None = None
if "memory" in mapping and mapping["memory"] is not None:
raw_stores = ensure_list(mapping.get("memory"))
stores: List[MemoryStoreConfig] = []
seen: set[str] = set()
for idx, item in enumerate(raw_stores):
store = MemoryStoreConfig.from_dict(item, path=extend_path(path, f"memory[{idx}]"))
if store.name in seen:
raise ConfigError(
f"duplicated memory store name '{store.name}'",
extend_path(path, f"memory[{idx}].name"),
)
seen.add(store.name)
stores.append(store)
memory_cfg = stores
start_nodes: List[str] = []
if "start" in mapping and mapping["start"] is not None:
start_value = mapping["start"]
if isinstance(start_value, str):
start_nodes = [start_value]
elif isinstance(start_value, list) and all(isinstance(item, str) for item in start_value):
seen = set()
start_nodes = []
for item in start_value:
if item not in seen:
seen.add(item)
start_nodes.append(item)
else:
raise ConfigError("start must be a string or list of strings if provided", extend_path(path, "start"))
end_nodes = None
if "end" in mapping and mapping["end"] is not None:
end_value = mapping["end"]
if isinstance(end_value, str):
end_nodes = [end_value]
elif isinstance(end_value, list) and all(isinstance(item, str) for item in end_value):
end_nodes = list(end_value)
else:
raise ConfigError("end must be a string or list of strings", extend_path(path, "end"))
definition = cls(
id=graph_id,
description=description,
log_level=log_level,
is_majority_voting=bool(is_majority) if is_majority is not None else False,
nodes=nodes,
edges=edges,
memory=memory_cfg,
organization=organization,
initial_instruction=initial_instruction,
start_nodes=start_nodes,
end_nodes=end_nodes,
path=path,
)
definition.validate()
return definition
def validate(self) -> None:
node_ids = [node.id for node in self.nodes]
counts = Counter(node_ids)
duplicates = [nid for nid, count in counts.items() if count > 1]
if duplicates:
dup_list = ", ".join(sorted(duplicates))
raise ConfigError(f"duplicate node ids detected: {dup_list}", extend_path(self.path, "nodes"))
node_set = set(node_ids)
for start_node in self.start_nodes:
if start_node not in node_set:
raise ConfigError(
f"start node '{start_node}' not defined in nodes",
extend_path(self.path, "start"),
)
for edge in self.edges:
if edge.source not in node_set:
raise ConfigError(
f"edge references unknown source node '{edge.source}'",
extend_path(self.path, f"edges->{edge.source}->{edge.target}"),
)
if edge.target not in node_set:
raise ConfigError(
f"edge references unknown target node '{edge.target}'",
extend_path(self.path, f"edges->{edge.source}->{edge.target}"),
)
store_names = {store.name for store in self.memory} if self.memory else set()
for node in self.nodes:
model = node.as_config(AgentConfig)
if model:
for attachment in model.memories:
if attachment.name not in store_names:
raise ConfigError(
f"memory reference '{attachment.name}' not defined in graph.memory",
attachment.path or extend_path(node.path, "config.memories"),
)
@dataclass
class DesignConfig(BaseConfig):
version: str
vars: Dict[str, Any]
graph: GraphDefinition
FIELD_SPECS = {
"version": ConfigFieldSpec(
name="version",
display_name="Configuration Version",
type_hint="str",
required=False,
default="0.0.0",
description="Configuration version number",
advance=True,
),
"vars": ConfigFieldSpec(
name="vars",
display_name="Global Variables",
type_hint="dict[str, Any]",
required=False,
default={},
description="Global variables that can be referenced via ${VAR}",
),
"graph": ConfigFieldSpec(
name="graph",
display_name="Graph Definition",
type_hint="GraphDefinition",
required=True,
description="Core graph definition",
child=GraphDefinition,
),
}
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str = "root") -> "DesignConfig":
mapping = require_mapping(data, path)
version = optional_str(mapping, "version", path) or "0.0.0"
vars_block = optional_dict(mapping, "vars", path) or {}
if "graph" not in mapping or mapping["graph"] is None:
raise ConfigError("graph section is required", extend_path(path, "graph"))
graph = GraphDefinition.from_dict(mapping["graph"], path=extend_path(path, "graph"))
return cls(version=version, vars=vars_block, graph=graph, path=path)

20
entity/configs/node/__init__.py Executable file
View File

@ -0,0 +1,20 @@
"""Node config conveniences."""
from .agent import AgentConfig, AgentRetryConfig
from .human import HumanConfig
from .subgraph import SubgraphConfig
from .passthrough import PassthroughConfig
from .python_runner import PythonRunnerConfig
from .node import Node
from .literal import LiteralNodeConfig
__all__ = [
"AgentConfig",
"AgentRetryConfig",
"HumanConfig",
"SubgraphConfig",
"PassthroughConfig",
"PythonRunnerConfig",
"LiteralNodeConfig",
"Node",
]

561
entity/configs/node/agent.py Executable file
View File

@ -0,0 +1,561 @@
"""Agent-specific configuration dataclasses."""
from dataclasses import dataclass, field, replace
from typing import Any, Dict, Iterable, List, Mapping, Sequence
try: # pragma: no cover - Python < 3.11 lacks BaseExceptionGroup
from builtins import BaseExceptionGroup as _BASE_EXCEPTION_GROUP_TYPE # type: ignore[attr-defined]
except ImportError: # pragma: no cover
_BASE_EXCEPTION_GROUP_TYPE = None # type: ignore[assignment]
from entity.enums import AgentInputMode
from schema_registry import iter_model_provider_schemas
from utils.strs import titleize
from entity.configs.base import (
BaseConfig,
ConfigError,
ConfigFieldSpec,
EnumOption,
optional_bool,
optional_dict,
optional_str,
require_mapping,
require_str,
extend_path,
)
from .memory import MemoryAttachmentConfig
from .thinking import ThinkingConfig
from entity.configs.node.tooling import ToolingConfig
DEFAULT_RETRYABLE_STATUS_CODES = [408, 409, 425, 429, 500, 502, 503, 504]
DEFAULT_RETRYABLE_EXCEPTION_TYPES = [
"RateLimitError",
"APITimeoutError",
"APIError",
"APIConnectionError",
"ServiceUnavailableError",
"TimeoutError",
"InternalServerError",
"RemoteProtocolError",
"TransportError",
"ConnectError",
"ConnectTimeout",
"ReadError",
"ReadTimeout",
]
DEFAULT_RETRYABLE_MESSAGE_SUBSTRINGS = [
"rate limit",
"temporarily unavailable",
"timeout",
"server disconnected",
"connection reset",
]
def _coerce_float(value: Any, *, field_path: str, minimum: float = 0.0) -> float:
if isinstance(value, (int, float)):
coerced = float(value)
else:
raise ConfigError("expected number", field_path)
if coerced < minimum:
raise ConfigError(f"value must be >= {minimum}", field_path)
return coerced
def _coerce_positive_int(value: Any, *, field_path: str, minimum: int = 1) -> int:
if isinstance(value, bool):
raise ConfigError("expected integer", field_path)
if isinstance(value, int):
coerced = value
else:
raise ConfigError("expected integer", field_path)
if coerced < minimum:
raise ConfigError(f"value must be >= {minimum}", field_path)
return coerced
def _coerce_str_list(value: Any, *, field_path: str) -> List[str]:
if value is None:
return []
if not isinstance(value, Sequence) or isinstance(value, (str, bytes)):
raise ConfigError("expected list of strings", field_path)
result: List[str] = []
for idx, item in enumerate(value):
if not isinstance(item, str):
raise ConfigError("expected list of strings", f"{field_path}[{idx}]")
result.append(item.strip())
return result
def _coerce_int_list(value: Any, *, field_path: str) -> List[int]:
if value is None:
return []
if not isinstance(value, Sequence) or isinstance(value, (str, bytes)):
raise ConfigError("expected list of integers", field_path)
ints: List[int] = []
for idx, item in enumerate(value):
if isinstance(item, bool) or not isinstance(item, int):
raise ConfigError("expected list of integers", f"{field_path}[{idx}]")
ints.append(item)
return ints
@dataclass
class AgentRetryConfig(BaseConfig):
enabled: bool = True
max_attempts: int = 5
min_wait_seconds: float = 1.0
max_wait_seconds: float = 6.0
retry_on_status_codes: List[int] = field(default_factory=lambda: list(DEFAULT_RETRYABLE_STATUS_CODES))
retry_on_exception_types: List[str] = field(default_factory=lambda: [name.lower() for name in DEFAULT_RETRYABLE_EXCEPTION_TYPES])
non_retry_exception_types: List[str] = field(default_factory=list)
retry_on_error_substrings: List[str] = field(default_factory=lambda: list(DEFAULT_RETRYABLE_MESSAGE_SUBSTRINGS))
FIELD_SPECS = {
"enabled": ConfigFieldSpec(
name="enabled",
display_name="Enable Retry",
type_hint="bool",
required=False,
default=True,
description="Toggle automatic retry for provider calls",
),
"max_attempts": ConfigFieldSpec(
name="max_attempts",
display_name="Max Attempts",
type_hint="int",
required=False,
default=5,
description="Maximum number of total attempts (initial call + retries)",
),
"min_wait_seconds": ConfigFieldSpec(
name="min_wait_seconds",
display_name="Min Wait Seconds",
type_hint="float",
required=False,
default=1.0,
description="Minimum backoff wait before retry",
advance=True,
),
"max_wait_seconds": ConfigFieldSpec(
name="max_wait_seconds",
display_name="Max Wait Seconds",
type_hint="float",
required=False,
default=6.0,
description="Maximum backoff wait before retry",
advance=True,
),
"retry_on_status_codes": ConfigFieldSpec(
name="retry_on_status_codes",
display_name="Retryable Status Codes",
type_hint="list[int]",
required=False,
description="HTTP status codes that should trigger a retry",
advance=True,
),
"retry_on_exception_types": ConfigFieldSpec(
name="retry_on_exception_types",
display_name="Retryable Exception Types",
type_hint="list[str]",
required=False,
description="Exception class names (case-insensitive) that should trigger retries",
advance=True,
),
"non_retry_exception_types": ConfigFieldSpec(
name="non_retry_exception_types",
display_name="Non-Retryable Exception Types",
type_hint="list[str]",
required=False,
description="Exception class names (case-insensitive) that should never retry",
advance=True,
),
"retry_on_error_substrings": ConfigFieldSpec(
name="retry_on_error_substrings",
display_name="Retryable Message Substrings",
type_hint="list[str]",
required=False,
description="Substring matches within exception messages that enable retry",
advance=True,
),
}
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "AgentRetryConfig":
mapping = require_mapping(data, path)
enabled = optional_bool(mapping, "enabled", path, default=True)
if enabled is None:
enabled = True
max_attempts = _coerce_positive_int(mapping.get("max_attempts", 5), field_path=extend_path(path, "max_attempts"))
min_wait = _coerce_float(mapping.get("min_wait_seconds", 1.0), field_path=extend_path(path, "min_wait_seconds"), minimum=0.0)
max_wait = _coerce_float(mapping.get("max_wait_seconds", 6.0), field_path=extend_path(path, "max_wait_seconds"), minimum=0.0)
if max_wait < min_wait:
raise ConfigError("max_wait_seconds must be >= min_wait_seconds", extend_path(path, "max_wait_seconds"))
status_codes = mapping.get("retry_on_status_codes")
if status_codes is None:
retry_status_codes = list(DEFAULT_RETRYABLE_STATUS_CODES)
else:
retry_status_codes = _coerce_int_list(status_codes, field_path=extend_path(path, "retry_on_status_codes"))
retry_types_raw = mapping.get("retry_on_exception_types")
if retry_types_raw is None:
retry_types = [name.lower() for name in DEFAULT_RETRYABLE_EXCEPTION_TYPES]
else:
retry_types = [value.lower() for value in _coerce_str_list(retry_types_raw, field_path=extend_path(path, "retry_on_exception_types")) if value]
non_retry_types = [value.lower() for value in _coerce_str_list(mapping.get("non_retry_exception_types"), field_path=extend_path(path, "non_retry_exception_types")) if value]
retry_substrings_raw = mapping.get("retry_on_error_substrings")
if retry_substrings_raw is None:
retry_substrings = list(DEFAULT_RETRYABLE_MESSAGE_SUBSTRINGS)
else:
retry_substrings = [
value.lower()
for value in _coerce_str_list(
retry_substrings_raw,
field_path=extend_path(path, "retry_on_error_substrings"),
)
if value
]
return cls(
enabled=enabled,
max_attempts=max_attempts,
min_wait_seconds=min_wait,
max_wait_seconds=max_wait,
retry_on_status_codes=retry_status_codes,
retry_on_exception_types=retry_types,
non_retry_exception_types=non_retry_types,
retry_on_error_substrings=retry_substrings,
path=path,
)
@property
def is_active(self) -> bool:
return self.enabled and self.max_attempts > 1
def should_retry(self, exc: BaseException) -> bool:
if not self.is_active:
return False
chain: List[tuple[BaseException, set[str], int | None, str]] = []
for error in self._iter_exception_chain(exc):
chain.append(
(
error,
self._exception_name_set(error),
self._extract_status_code(error),
str(error).lower(),
)
)
if self.non_retry_exception_types:
for _, names, _, _ in chain:
if any(name in names for name in self.non_retry_exception_types):
return False
if self.retry_on_exception_types:
for _, names, _, _ in chain:
if any(name in names for name in self.retry_on_exception_types):
return True
if self.retry_on_status_codes:
for _, _, status_code, _ in chain:
if status_code is not None and status_code in self.retry_on_status_codes:
return True
if self.retry_on_error_substrings:
for _, _, _, message in chain:
if message and any(substr in message for substr in self.retry_on_error_substrings):
return True
return False
def _exception_name_set(self, exc: BaseException) -> set[str]:
names: set[str] = set()
for cls in exc.__class__.mro():
names.add(cls.__name__.lower())
names.add(f"{cls.__module__}.{cls.__name__}".lower())
return names
def _extract_status_code(self, exc: BaseException) -> int | None:
for attr in ("status_code", "http_status", "status", "statusCode"):
value = getattr(exc, attr, None)
if isinstance(value, int):
return value
response = getattr(exc, "response", None)
if response is not None:
for attr in ("status_code", "status", "statusCode"):
value = getattr(response, attr, None)
if isinstance(value, int):
return value
return None
def _iter_exception_chain(self, exc: BaseException) -> Iterable[BaseException]:
seen: set[int] = set()
stack: List[BaseException] = [exc]
while stack:
current = stack.pop()
if id(current) in seen:
continue
seen.add(id(current))
yield current
linked: List[BaseException] = []
cause = getattr(current, "__cause__", None)
context = getattr(current, "__context__", None)
if isinstance(cause, BaseException):
linked.append(cause)
if isinstance(context, BaseException):
linked.append(context)
if _BASE_EXCEPTION_GROUP_TYPE is not None and isinstance(current, _BASE_EXCEPTION_GROUP_TYPE):
for exc_item in getattr(current, "exceptions", None) or ():
if isinstance(exc_item, BaseException):
linked.append(exc_item)
stack.extend(linked)
@dataclass
class AgentConfig(BaseConfig):
provider: str
base_url: str
name: str
role: str | None = None
api_key: str | None = None
params: Dict[str, Any] = field(default_factory=dict)
retry: AgentRetryConfig | None = None
input_mode: AgentInputMode = AgentInputMode.MESSAGES
tooling: List[ToolingConfig] = field(default_factory=list)
thinking: ThinkingConfig | None = None
memories: List[MemoryAttachmentConfig] = field(default_factory=list)
# Runtime attributes (attached dynamically)
token_tracker: Any | None = field(default=None, init=False, repr=False)
node_id: str | None = field(default=None, init=False, repr=False)
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "AgentConfig":
mapping = require_mapping(data, path)
provider = require_str(mapping, "provider", path)
base_url = optional_str(mapping, "base_url", path)
name_value = mapping.get("name")
if isinstance(name_value, str) and name_value.strip():
model_name = name_value.strip()
else:
raise ConfigError("model.name must be a non-empty string", extend_path(path, "name"))
role = optional_str(mapping, "role", path)
api_key = optional_str(mapping, "api_key", path)
params = optional_dict(mapping, "params", path) or {}
raw_input_mode = optional_str(mapping, "input_mode", path)
input_mode = AgentInputMode.MESSAGES
if raw_input_mode:
try:
input_mode = AgentInputMode(raw_input_mode.strip().lower())
except ValueError as exc:
raise ConfigError(
"model.input_mode must be 'prompt' or 'messages'",
extend_path(path, "input_mode"),
) from exc
tooling_cfg: List[ToolingConfig] = []
if "tooling" in mapping and mapping["tooling"] is not None:
raw_tooling = mapping["tooling"]
if not isinstance(raw_tooling, list):
raise ConfigError("tooling must be a list", extend_path(path, "tooling"))
for idx, item in enumerate(raw_tooling):
tooling_cfg.append(
ToolingConfig.from_dict(item, path=extend_path(path, f"tooling[{idx}]"))
)
thinking_cfg = None
if "thinking" in mapping and mapping["thinking"] is not None:
thinking_cfg = ThinkingConfig.from_dict(mapping["thinking"], path=extend_path(path, "thinking"))
memories_cfg: List[MemoryAttachmentConfig] = []
if "memories" in mapping and mapping["memories"] is not None:
raw_memories = mapping["memories"]
if not isinstance(raw_memories, list):
raise ConfigError("memories must be a list", extend_path(path, "memories"))
for idx, item in enumerate(raw_memories):
memories_cfg.append(
MemoryAttachmentConfig.from_dict(item, path=extend_path(path, f"memories[{idx}]"))
)
retry_cfg = None
if "retry" in mapping and mapping["retry"] is not None:
retry_cfg = AgentRetryConfig.from_dict(mapping["retry"], path=extend_path(path, "retry"))
return cls(
provider=provider,
base_url=base_url,
name=model_name,
role=role,
api_key=api_key,
params=params,
tooling=tooling_cfg,
thinking=thinking_cfg,
memories=memories_cfg,
retry=retry_cfg,
input_mode=input_mode,
path=path,
)
FIELD_SPECS = {
"name": ConfigFieldSpec(
name="name",
display_name="Model Name",
type_hint="str",
required=True,
description="Specific model name e.g. gpt-4o",
),
"role": ConfigFieldSpec(
name="role",
display_name="System Prompt",
type_hint="text",
required=False,
description="Model system prompt",
),
"provider": ConfigFieldSpec(
name="provider",
display_name="Model Provider",
type_hint="str",
required=True,
description="Name of a registered provider (openai, gemini, etc.) that selects the underlying client adapter.",
default="openai",
),
"base_url": ConfigFieldSpec(
name="base_url",
display_name="Base URL",
type_hint="str",
required=False,
description="Override the provider's default endpoint; leave empty to use the built-in base URL.",
advance=True,
default="${BASE_URL}",
),
"api_key": ConfigFieldSpec(
name="api_key",
display_name="API Key",
type_hint="str",
required=False,
description="Credential consumed by the provider client; reference an env var such as ${API_KEY} that matches the selected provider.",
advance=True,
default="${API_KEY}",
),
"params": ConfigFieldSpec(
name="params",
display_name="Call Parameters",
type_hint="dict[str, Any]",
required=False,
default={},
description="Call parameters (temperature, top_p, etc.)",
advance=True,
),
# "input_mode": ConfigFieldSpec( # currently, many features depend on messages mode, so hide this and force messages
# name="input_mode",
# display_name="Input Mode",
# type_hint="enum:AgentInputMode",
# required=False,
# default=AgentInputMode.MESSAGES.value,
# description="Model input mode: messages (default) or prompt",
# enum=[item.value for item in AgentInputMode],
# advance=True,
# enum_options=enum_options_for(AgentInputMode),
# ),
"tooling": ConfigFieldSpec(
name="tooling",
display_name="Tool Configuration",
type_hint="list[ToolingConfig]",
required=False,
description="Bound tool configuration list",
child=ToolingConfig,
advance=True,
),
"thinking": ConfigFieldSpec(
name="thinking",
display_name="Thinking Configuration",
type_hint="ThinkingConfig",
required=False,
description="Thinking process configuration",
child=ThinkingConfig,
advance=True,
),
"memories": ConfigFieldSpec(
name="memories",
display_name="Memory Attachments",
type_hint="list[MemoryAttachmentConfig]",
required=False,
description="Associated memory references",
child=MemoryAttachmentConfig,
advance=True,
),
"retry": ConfigFieldSpec(
name="retry",
display_name="Retry Policy",
type_hint="AgentRetryConfig",
required=False,
description="Automatic retry policy for this model",
child=AgentRetryConfig,
advance=True,
),
}
@classmethod
def field_specs(cls) -> Dict[str, ConfigFieldSpec]:
specs = super().field_specs()
provider_spec = specs.get("provider")
if provider_spec:
enum_spec = cls._apply_provider_enum(provider_spec)
specs["provider"] = enum_spec
return specs
@staticmethod
def _apply_provider_enum(provider_spec: ConfigFieldSpec) -> ConfigFieldSpec:
provider_names, metadata = AgentConfig._provider_registry_snapshot()
if not provider_names:
return provider_spec
enum_options: List[EnumOption] = []
for name in provider_names:
meta = metadata.get(name) or {}
label = meta.get("label") or titleize(name)
enum_options.append(
EnumOption(
value=name,
label=label,
description=meta.get("summary"),
)
)
default_value = provider_spec.default
if not default_value or default_value not in provider_names:
default_value = AgentConfig._preferred_provider_default(provider_names)
return replace(
provider_spec,
enum=provider_names,
enum_options=enum_options,
default=default_value,
)
@staticmethod
def _preferred_provider_default(provider_names: List[str]) -> str:
if "openai" in provider_names:
return "openai"
return provider_names[0]
@staticmethod
def _provider_registry_snapshot() -> tuple[List[str], Dict[str, Dict[str, Any]]]:
specs = iter_model_provider_schemas()
names = list(specs.keys())
metadata: Dict[str, Dict[str, Any]] = {}
for name, spec in specs.items():
metadata[name] = {
"label": spec.label,
"summary": spec.summary,
**(spec.metadata or {}),
}
return names, metadata

29
entity/configs/node/human.py Executable file
View File

@ -0,0 +1,29 @@
"""Human node configuration."""
from dataclasses import dataclass
from typing import Any, Mapping
from entity.configs.base import BaseConfig, ConfigFieldSpec, optional_str, require_mapping
@dataclass
class HumanConfig(BaseConfig):
description: str | None = None
@classmethod
def from_dict(cls, data: Mapping[str, Any] | None, *, path: str) -> "HumanConfig":
if data is None:
return cls(description=None, path=path)
mapping = require_mapping(data, path)
description = optional_str(mapping, "description", path)
return cls(description=description, path=path)
FIELD_SPECS = {
"description": ConfigFieldSpec(
name="description",
display_name="Human Task Description",
type_hint="text",
required=False,
description="Description of the task for human to complete",
)
}

69
entity/configs/node/literal.py Executable file
View File

@ -0,0 +1,69 @@
"""Configuration for literal nodes."""
from dataclasses import dataclass
from typing import Mapping, Any
from entity.configs.base import (
BaseConfig,
ConfigError,
ConfigFieldSpec,
EnumOption,
optional_str,
require_mapping,
require_str,
)
from entity.messages import MessageRole
@dataclass
class LiteralNodeConfig(BaseConfig):
"""Config describing the literal payload emitted by the node."""
content: str = ""
role: MessageRole = MessageRole.USER
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "LiteralNodeConfig":
mapping = require_mapping(data, path)
content = require_str(mapping, "content", path)
if not content:
raise ConfigError("content cannot be empty", f"{path}.content")
role_value = optional_str(mapping, "role", path)
role = MessageRole.USER
if role_value:
normalized = role_value.strip().lower()
if normalized not in (MessageRole.USER.value, MessageRole.ASSISTANT.value):
raise ConfigError("role must be 'user' or 'assistant'", f"{path}.role")
role = MessageRole(normalized)
return cls(content=content, role=role, path=path)
def validate(self) -> None:
if not self.content:
raise ConfigError("content cannot be empty", f"{self.path}.content")
if self.role not in (MessageRole.USER, MessageRole.ASSISTANT):
raise ConfigError("role must be 'user' or 'assistant'", f"{self.path}.role")
FIELD_SPECS = {
"content": ConfigFieldSpec(
name="content",
display_name="Literal Content",
type_hint="text",
required=True,
description="Plain text emitted whenever the node executes.",
),
"role": ConfigFieldSpec(
name="role",
display_name="Message Role",
type_hint="str",
required=False,
default=MessageRole.USER.value,
enum=[MessageRole.USER.value, MessageRole.ASSISTANT.value],
enum_options=[
EnumOption(value=MessageRole.USER.value, label="user"),
EnumOption(value=MessageRole.ASSISTANT.value, label="assistant"),
],
description="Select whether the literal message should appear as a user or assistant entry.",
),
}

79
entity/configs/node/loop_counter.py Executable file
View File

@ -0,0 +1,79 @@
"""Configuration for loop counter guard nodes."""
from dataclasses import dataclass
from typing import Mapping, Any, Optional
from entity.configs.base import (
BaseConfig,
ConfigError,
ConfigFieldSpec,
require_mapping,
extend_path,
optional_str,
)
@dataclass
class LoopCounterConfig(BaseConfig):
"""Configuration schema for the loop counter node type."""
max_iterations: int = 10
reset_on_emit: bool = True
message: Optional[str] = None
@classmethod
def from_dict(cls, data: Mapping[str, Any] | None, *, path: str) -> "LoopCounterConfig":
mapping = require_mapping(data or {}, path)
max_iterations_raw = mapping.get("max_iterations", 10)
try:
max_iterations = int(max_iterations_raw)
except (TypeError, ValueError) as exc: # pragma: no cover - defensive
raise ConfigError(
"max_iterations must be an integer",
extend_path(path, "max_iterations"),
) from exc
if max_iterations < 1:
raise ConfigError("max_iterations must be >= 1", extend_path(path, "max_iterations"))
reset_on_emit = bool(mapping.get("reset_on_emit", True))
message = optional_str(mapping, "message", path)
return cls(
max_iterations=max_iterations,
reset_on_emit=reset_on_emit,
message=message,
path=path,
)
def validate(self) -> None:
if self.max_iterations < 1:
raise ConfigError("max_iterations must be >= 1", extend_path(self.path, "max_iterations"))
FIELD_SPECS = {
"max_iterations": ConfigFieldSpec(
name="max_iterations",
display_name="Maximum Iterations",
type_hint="int",
required=True,
default=10,
description="How many times the loop can run before this node emits an output.",
),
"reset_on_emit": ConfigFieldSpec(
name="reset_on_emit",
display_name="Reset After Emit",
type_hint="bool",
required=False,
default=True,
description="Whether to reset the internal counter after reaching the limit.",
advance=True,
),
"message": ConfigFieldSpec(
name="message",
display_name="Release Message",
type_hint="text",
required=False,
description="Optional text sent downstream once the iteration cap is reached.",
advance=True,
),
}

464
entity/configs/node/memory.py Executable file
View File

@ -0,0 +1,464 @@
"""Memory-related configuration dataclasses."""
from dataclasses import dataclass, field, replace
from typing import Any, Dict, List, Mapping
from entity.enums import AgentExecFlowStage
from entity.enum_options import enum_options_for, enum_options_from_values
from schema_registry import (
SchemaLookupError,
get_memory_store_schema,
iter_memory_store_schemas,
)
from entity.configs.base import (
BaseConfig,
ConfigError,
ConfigFieldSpec,
ChildKey,
ensure_list,
optional_dict,
optional_str,
require_mapping,
require_str,
extend_path,
)
@dataclass
class EmbeddingConfig(BaseConfig):
provider: str
model: str
api_key: str | None = None
base_url: str | None = None
params: Dict[str, Any] = field(default_factory=dict)
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "EmbeddingConfig":
mapping = require_mapping(data, path)
provider = require_str(mapping, "provider", path)
model = require_str(mapping, "model", path)
api_key = optional_str(mapping, "api_key", path)
base_url = optional_str(mapping, "base_url", path)
params = optional_dict(mapping, "params", path) or {}
return cls(provider=provider, model=model, api_key=api_key, base_url=base_url, params=params, path=path)
FIELD_SPECS = {
"provider": ConfigFieldSpec(
name="provider",
display_name="Embedding Provider",
type_hint="str",
required=True,
default="openai",
description="Embedding provider",
),
"model": ConfigFieldSpec(
name="model",
display_name="Embedding Model",
type_hint="str",
required=True,
default="text-embedding-3-small",
description="Embedding model name",
),
"api_key": ConfigFieldSpec(
name="api_key",
display_name="API Key",
type_hint="str",
required=False,
description="API key",
default="${API_KEY}",
advance=True,
),
"base_url": ConfigFieldSpec(
name="base_url",
display_name="Base URL",
type_hint="str",
required=False,
description="Custom Base URL",
default="${BASE_URL}",
advance=True,
),
"params": ConfigFieldSpec(
name="params",
display_name="Custom Parameters",
type_hint="dict[str, Any]",
required=False,
default={},
description="Embedding parameters (temperature, etc.)",
advance=True,
),
}
@dataclass
class FileSourceConfig(BaseConfig):
source_path: str
file_types: List[str] | None = None
recursive: bool = True
encoding: str = "utf-8"
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "FileSourceConfig":
mapping = require_mapping(data, path)
file_path = require_str(mapping, "path", path)
file_types_value = mapping.get("file_types")
file_types: List[str] | None = None
if file_types_value is not None:
items = ensure_list(file_types_value)
normalized: List[str] = []
for idx, item in enumerate(items):
if not isinstance(item, str):
raise ConfigError("file_types entries must be strings", extend_path(path, f"file_types[{idx}]") )
normalized.append(item)
file_types = normalized
recursive_value = mapping.get("recursive", True)
if not isinstance(recursive_value, bool):
raise ConfigError("recursive must be boolean", extend_path(path, "recursive"))
encoding = optional_str(mapping, "encoding", path) or "utf-8"
return cls(source_path=file_path, file_types=file_types, recursive=recursive_value, encoding=encoding, path=path)
FIELD_SPECS = {
"path": ConfigFieldSpec(
name="path",
display_name="File/Directory Path",
type_hint="str",
required=True,
description="Path to file/directory to be indexed",
),
"file_types": ConfigFieldSpec(
name="file_types",
display_name="File Type Filter",
type_hint="list[str]",
required=False,
description="List of file type suffixes to limit (e.g. .md, .txt)",
),
"recursive": ConfigFieldSpec(
name="recursive",
display_name="Recursive Subdirectories",
type_hint="bool",
required=False,
default=True,
description="Whether to include subdirectories recursively",
advance=True,
),
"encoding": ConfigFieldSpec(
name="encoding",
display_name="File Encoding",
type_hint="str",
required=False,
default="utf-8",
description="Encoding used to read files",
advance=True,
),
}
@dataclass
class SimpleMemoryConfig(BaseConfig):
memory_path: str | None = None
embedding: EmbeddingConfig | None = None
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "SimpleMemoryConfig":
mapping = require_mapping(data, path)
memory_path = optional_str(mapping, "memory_path", path)
embedding_cfg = None
if "embedding" in mapping and mapping["embedding"] is not None:
embedding_cfg = EmbeddingConfig.from_dict(mapping["embedding"], path=extend_path(path, "embedding"))
return cls(memory_path=memory_path, embedding=embedding_cfg, path=path)
FIELD_SPECS = {
"memory_path": ConfigFieldSpec(
name="memory_path",
display_name="Memory File Path",
type_hint="str",
required=False,
description="Simple memory file path",
advance=True,
),
"embedding": ConfigFieldSpec(
name="embedding",
display_name="Embedding Configuration",
type_hint="EmbeddingConfig",
required=False,
description="Optional embedding configuration",
child=EmbeddingConfig,
),
}
@dataclass
class FileMemoryConfig(BaseConfig):
index_path: str | None = None
file_sources: List[FileSourceConfig] = field(default_factory=list)
embedding: EmbeddingConfig | None = None
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "FileMemoryConfig":
mapping = require_mapping(data, path)
sources_raw = ensure_list(mapping.get("file_sources"))
if not sources_raw:
raise ConfigError("file_sources must contain at least one entry", extend_path(path, "file_sources"))
sources: List[FileSourceConfig] = []
for idx, item in enumerate(sources_raw):
sources.append(FileSourceConfig.from_dict(item, path=extend_path(path, f"file_sources[{idx}]")))
index_path = optional_str(mapping, "index_path", path)
if index_path is None:
index_path = optional_str(mapping, "memory_path", path)
embedding_cfg = None
if "embedding" in mapping and mapping["embedding"] is not None:
embedding_cfg = EmbeddingConfig.from_dict(mapping["embedding"], path=extend_path(path, "embedding"))
return cls(index_path=index_path, file_sources=sources, embedding=embedding_cfg, path=path)
FIELD_SPECS = {
"index_path": ConfigFieldSpec(
name="index_path",
display_name="Index Path",
type_hint="str",
required=False,
description="Vector index storage path",
advance=True,
),
"file_sources": ConfigFieldSpec(
name="file_sources",
display_name="File Source List",
type_hint="list[FileSourceConfig]",
required=True,
description="List of file sources to ingest",
child=FileSourceConfig,
),
"embedding": ConfigFieldSpec(
name="embedding",
display_name="Embedding Configuration",
type_hint="EmbeddingConfig",
required=False,
description="Embedding used for file memory",
child=EmbeddingConfig,
),
}
@dataclass
class BlackboardMemoryConfig(BaseConfig):
memory_path: str | None = None
max_items: int = 1000
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "BlackboardMemoryConfig":
mapping = require_mapping(data, path)
memory_path = optional_str(mapping, "memory_path", path)
max_items_value = mapping.get("max_items", 1000)
if not isinstance(max_items_value, int) or max_items_value <= 0:
raise ConfigError("max_items must be a positive integer", extend_path(path, "max_items"))
return cls(memory_path=memory_path, max_items=max_items_value, path=path)
FIELD_SPECS = {
"memory_path": ConfigFieldSpec(
name="memory_path",
display_name="Blackboard Path",
type_hint="str",
required=False,
description="JSON path for blackboard memory writing. Pass 'auto' to auto-create in working directory, valid for this run only",
default="auto",
advance=True,
),
"max_items": ConfigFieldSpec(
name="max_items",
display_name="Maximum Items",
type_hint="int",
required=False,
default=1000,
description="Maximum number of memory items to retain (trimmed by time)",
advance=True,
),
}
@dataclass
class MemoryStoreConfig(BaseConfig):
name: str
type: str
config: BaseConfig | None = None
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "MemoryStoreConfig":
mapping = require_mapping(data, path)
name = require_str(mapping, "name", path)
store_type = require_str(mapping, "type", path)
try:
schema = get_memory_store_schema(store_type)
except SchemaLookupError as exc:
raise ConfigError(f"unsupported memory store type '{store_type}'", extend_path(path, "type")) from exc
if "config" not in mapping or mapping["config"] is None:
raise ConfigError("memory store requires config block", extend_path(path, "config"))
config_obj = schema.config_cls.from_dict(mapping["config"], path=extend_path(path, "config"))
return cls(name=name, type=store_type, config=config_obj, path=path)
def require_payload(self) -> BaseConfig:
if not self.config:
raise ConfigError("memory store payload missing", extend_path(self.path, "config"))
return self.config
FIELD_SPECS = {
"name": ConfigFieldSpec(
name="name",
display_name="Store Name",
type_hint="str",
required=True,
description="Unique name of the memory store",
),
"type": ConfigFieldSpec(
name="type",
display_name="Store Type",
type_hint="str",
required=True,
description="Memory store type",
),
"config": ConfigFieldSpec(
name="config",
display_name="Store Configuration",
type_hint="object",
required=True,
description="Schema required by the selected store type (simple/file/blackboard/etc.), following that type's required keys.",
),
}
@classmethod
def child_routes(cls) -> Dict[ChildKey, type[BaseConfig]]:
return {
ChildKey(field="config", value=name): schema.config_cls
for name, schema in iter_memory_store_schemas().items()
}
@classmethod
def field_specs(cls) -> Dict[str, ConfigFieldSpec]:
specs = super().field_specs()
type_spec = specs.get("type")
if type_spec:
registrations = iter_memory_store_schemas()
names = list(registrations.keys())
descriptions = {name: schema.summary for name, schema in registrations.items()}
specs["type"] = replace(
type_spec,
enum=names,
enum_options=enum_options_from_values(names, descriptions, preserve_label_case=True),
)
return specs
@dataclass
class MemoryAttachmentConfig(BaseConfig):
name: str
retrieve_stage: List[AgentExecFlowStage] | None = None
top_k: int = 3
similarity_threshold: float = -1.0
read: bool = True
write: bool = True
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "MemoryAttachmentConfig":
mapping = require_mapping(data, path)
name = require_str(mapping, "name", path)
stages_raw = mapping.get("retrieve_stage")
stages: List[AgentExecFlowStage] | None = None
if stages_raw is not None:
stage_list = ensure_list(stages_raw)
parsed: List[AgentExecFlowStage] = []
for idx, item in enumerate(stage_list):
try:
parsed.append(AgentExecFlowStage(item))
except ValueError as exc:
raise ConfigError(
f"retrieve_stage entries must be one of {[stage.value for stage in AgentExecFlowStage]}",
extend_path(path, f"retrieve_stage[{idx}]"),
) from exc
stages = parsed
top_k_value = mapping.get("top_k", 3)
if not isinstance(top_k_value, int) or top_k_value <= 0:
raise ConfigError("top_k must be a positive integer", extend_path(path, "top_k"))
threshold_value = mapping.get("similarity_threshold", -1.0)
if not isinstance(threshold_value, (int, float)):
raise ConfigError("similarity_threshold must be numeric", extend_path(path, "similarity_threshold"))
read_value = mapping.get("read", True)
if not isinstance(read_value, bool):
raise ConfigError("read must be boolean", extend_path(path, "read"))
write_value = mapping.get("write", True)
if not isinstance(write_value, bool):
raise ConfigError("write must be boolean", extend_path(path, "write"))
return cls(
name=name,
retrieve_stage=stages,
top_k=top_k_value,
similarity_threshold=float(threshold_value),
read=read_value,
write=write_value,
path=path,
)
FIELD_SPECS = {
"name": ConfigFieldSpec(
name="name",
display_name="Memory Name",
type_hint="str",
required=True,
description="Name of the referenced memory store",
),
"retrieve_stage": ConfigFieldSpec(
name="retrieve_stage",
display_name="Retrieve Stage",
type_hint="list[AgentExecFlowStage]",
required=False,
description="Execution stages when memory retrieval occurs. If not set, defaults to all stages. NOTE: this config is related to thinking, if the thinking module is not used, this config has only effect on `gen` stage.",
enum=[stage.value for stage in AgentExecFlowStage],
enum_options=enum_options_for(AgentExecFlowStage),
),
"top_k": ConfigFieldSpec(
name="top_k",
display_name="Top K",
type_hint="int",
required=False,
default=3,
description="Number of items to retrieve",
advance=True,
),
"similarity_threshold": ConfigFieldSpec(
name="similarity_threshold",
display_name="Similarity Threshold",
type_hint="float",
required=False,
default=-1.0,
description="Similarity threshold (-1 means no similarity threshold filter)",
advance=True,
),
"read": ConfigFieldSpec(
name="read",
display_name="Allow Read",
type_hint="bool",
required=False,
default=True,
description="Whether to read this memory during execution",
advance=True,
),
"write": ConfigFieldSpec(
name="write",
display_name="Allow Write",
type_hint="bool",
required=False,
default=True,
description="Whether to write back to this memory after execution",
advance=True,
),
}

459
entity/configs/node/node.py Executable file
View File

@ -0,0 +1,459 @@
"""Node configuration dataclasses."""
from dataclasses import dataclass, field, replace
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple
from entity.messages import Message, MessageRole
from schema_registry import (
SchemaLookupError,
get_node_schema,
iter_node_schemas,
)
from entity.configs.base import (
BaseConfig,
ConfigError,
ConfigFieldSpec,
EnumOption,
ChildKey,
ensure_list,
optional_str,
require_mapping,
require_str,
extend_path,
)
from entity.configs.edge.edge_condition import EdgeConditionConfig
from entity.configs.edge.edge_processor import EdgeProcessorConfig
from entity.configs.edge.dynamic_edge_config import DynamicEdgeConfig
from entity.configs.node.agent import AgentConfig
from entity.configs.node.human import HumanConfig
from entity.configs.node.tooling import FunctionToolConfig
NodePayload = Message
@dataclass
class EdgeLink:
target: "Node"
config: Dict[str, Any] = field(default_factory=dict)
trigger: bool = True
condition: str = "true"
condition_config: EdgeConditionConfig | None = None
condition_type: str | None = None
condition_metadata: Dict[str, Any] = field(default_factory=dict)
triggered: bool = False
carry_data: bool = True
keep_message: bool = False
clear_context: bool = False
clear_kept_context: bool = False
condition_manager: Any = None
process_config: EdgeProcessorConfig | None = None
process_type: str | None = None
process_metadata: Dict[str, Any] = field(default_factory=dict)
payload_processor: Any = None
dynamic_config: DynamicEdgeConfig | None = None
def __post_init__(self) -> None:
self.config = dict(self.config or {})
@dataclass
class Node(BaseConfig):
id: str
type: str
description: str | None = None
# keep_context: bool = False
context_window: int = 0
vars: Dict[str, Any] = field(default_factory=dict)
config: BaseConfig | None = None
# dynamic configuration has been moved to edges (DynamicEdgeConfig)
input: List[Message] = field(default_factory=list)
output: List[NodePayload] = field(default_factory=list)
# Runtime flag for explicit graph start nodes
start_triggered: bool = False
predecessors: List["Node"] = field(default_factory=list, repr=False)
successors: List["Node"] = field(default_factory=list, repr=False)
_outgoing_edges: List[EdgeLink] = field(default_factory=list, repr=False)
FIELD_SPECS = {
"id": ConfigFieldSpec(
name="id",
display_name="Node ID",
type_hint="str",
required=True,
description="Unique node identifier",
),
"type": ConfigFieldSpec(
name="type",
display_name="Node Type",
type_hint="str",
required=True,
description="Select a node type registered in node.registry (agent, human, python_runner, etc.); it determines the config schema.",
),
"description": ConfigFieldSpec(
name="description",
display_name="Node Description",
type_hint="str",
required=False,
advance=True,
description="Short summary shown in consoles/logs to explain this node's role or prompt context.",
),
# "keep_context": ConfigFieldSpec(
# name="keep_context",
# display_name="Preserve Context",
# type_hint="bool",
# required=False,
# default=False,
# description="Nodes clear their context by default; set to True to keep context data after execution.",
# ),
"context_window": ConfigFieldSpec(
name="context_window",
display_name="Context Window Size",
type_hint="int",
required=False,
default=0,
description="Number of context messages accessible during node execution. 0 means clear all context except messages with keep_message=True, -1 means unlimited, other values represent the number of context messages to keep besides those with keep_message=True.",
# advance=True,
),
"config": ConfigFieldSpec(
name="config",
display_name="Node Configuration",
type_hint="object",
required=True,
description="Configuration object required by the chosen node type (see Schema API for the supported fields).",
),
# Dynamic execution configuration has been moved to edges (DynamicEdgeConfig)
}
@classmethod
def child_routes(cls) -> Dict[ChildKey, type[BaseConfig]]:
routes: Dict[ChildKey, type[BaseConfig]] = {}
for name, schema in iter_node_schemas().items():
routes[ChildKey(field="config", value=name)] = schema.config_cls
return routes
@classmethod
def field_specs(cls) -> Dict[str, ConfigFieldSpec]:
specs = super().field_specs()
type_spec = specs.get("type")
if type_spec:
registrations = iter_node_schemas()
specs["type"] = replace(
type_spec,
enum=list(registrations.keys()),
enum_options=[
EnumOption(
value=name,
label=name,
description=schema.summary or "No description provided for this node type",
)
for name, schema in registrations.items()
],
)
return specs
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "Node":
mapping = require_mapping(data, path)
node_id = require_str(mapping, "id", path)
node_type = require_str(mapping, "type", path)
try:
schema = get_node_schema(node_type)
except SchemaLookupError as exc:
raise ConfigError(
f"unsupported node type '{node_type}'",
extend_path(path, "type"),
) from exc
description = optional_str(mapping, "description", path)
# keep_context = bool(mapping.get("keep_context", False))
context_window = int(mapping.get("context_window", 0))
input_value = ensure_list(mapping.get("input"))
output_value = ensure_list(mapping.get("output"))
input_messages: List[Message] = []
for value in input_value:
if isinstance(value, dict) and "role" in value:
input_messages.append(Message.from_dict(value))
elif isinstance(value, Message):
input_messages.append(value)
else:
input_messages.append(Message(role=MessageRole.USER, content=str(value)))
if "config" not in mapping or mapping["config"] is None:
raise ConfigError("node config block required", extend_path(path, "config"))
config_obj = schema.config_cls.from_dict(
mapping["config"], path=extend_path(path, "config")
)
formatted_output: List[NodePayload] = []
for value in output_value:
if isinstance(value, dict) and "role" in value:
formatted_output.append(Message.from_dict(value))
elif isinstance(value, Message):
formatted_output.append(value)
else:
formatted_output.append(
Message(role=MessageRole.ASSISTANT, content=str(value))
)
# Dynamic configuration parsing removed - dynamic is now on edges
node = cls(
id=node_id,
type=node_type,
description=description,
input=input_messages,
output=formatted_output,
# keep_context=keep_context,
context_window=context_window,
vars={},
config=config_obj,
path=path,
)
node.validate()
return node
def append_input(self, message: Message) -> None:
self.input.append(message)
def append_output(self, payload: NodePayload) -> None:
self.output.append(payload)
def clear_input(self, *, preserve_kept: bool = False, context_window: int = 0) -> int:
"""Clear queued inputs according to the node's context window semantics."""
if not preserve_kept:
self.input = []
return len(self.input)
if context_window < 0:
return len(self.input)
if context_window == 0:
self.input = [message for message in self.input if getattr(message, "keep", False)]
return len(self.input)
# context_window > 0 => retain the newest messages up to the specified
# capacity, but never drop messages flagged with keep=True. Those kept
# messages still count toward the window, effectively consuming slots that
# would otherwise be available for non-kept inputs.
keep_count = sum(1 for message in self.input if getattr(message, "keep", False))
allowed_non_keep = max(0, context_window - keep_count)
non_keep_total = sum(1 for message in self.input if not getattr(message, "keep", False))
non_keep_to_drop = max(0, non_keep_total - allowed_non_keep)
trimmed_inputs: List[Message] = []
for message in self.input:
if getattr(message, "keep", False):
trimmed_inputs.append(message)
continue
if non_keep_to_drop > 0:
non_keep_to_drop -= 1
continue
trimmed_inputs.append(message)
self.input = trimmed_inputs
return len(self.input)
def clear_inputs_by_flag(self, *, drop_non_keep: bool, drop_keep: bool) -> Tuple[int, int]:
"""Clear queued inputs according to keep markers."""
if not drop_non_keep and not drop_keep:
return 0, 0
remaining: List[Message] = []
removed_non_keep = 0
removed_keep = 0
for message in self.input:
is_keep = message.keep
if is_keep and drop_keep:
removed_keep += 1
continue
if not is_keep and drop_non_keep:
removed_non_keep += 1
continue
remaining.append(message)
if removed_non_keep or removed_keep:
self.input = remaining
return removed_non_keep, removed_keep
def validate(self) -> None:
if not self.config:
raise ConfigError("node configuration missing", extend_path(self.path, "config"))
if hasattr(self.config, "validate"):
self.config.validate()
@property
def node_type(self) -> str:
return self.type
@property
def model_name(self) -> Optional[str]:
agent = self.as_config(AgentConfig)
if not agent:
return None
return agent.name
@property
def role(self) -> Optional[str]:
agent = self.as_config(AgentConfig)
if agent:
return agent.role
human = self.as_config(HumanConfig)
if human:
return human.description
return None
@property
def tools(self) -> List[Any]:
agent = self.as_config(AgentConfig)
if agent and agent.tooling:
all_tools: List[Any] = []
for tool_config in agent.tooling:
func_cfg = tool_config.as_config(FunctionToolConfig)
if func_cfg:
all_tools.extend(func_cfg.tools)
return all_tools
return []
@property
def memories(self) -> List[Any]:
agent = self.as_config(AgentConfig)
if agent:
return list(agent.memories)
return []
@property
def params(self) -> Dict[str, Any]:
agent = self.as_config(AgentConfig)
if agent:
return dict(agent.params)
return {}
@property
def base_url(self) -> Optional[str]:
agent = self.as_config(AgentConfig)
if agent:
return agent.base_url
return None
def add_successor(self, node: "Node", edge_config: Optional[Dict[str, Any]] = None) -> None:
if node not in self.successors:
self.successors.append(node)
payload = dict(edge_config or {})
existing = next((link for link in self._outgoing_edges if link.target is node), None)
trigger = bool(payload.get("trigger", True)) if payload else True
carry_data = bool(payload.get("carry_data", True)) if payload else True
keep_message = bool(payload.get("keep_message", False)) if payload else False
clear_context = bool(payload.get("clear_context", False)) if payload else False
clear_kept_context = bool(payload.get("clear_kept_context", False)) if payload else False
condition_config = payload.pop("condition_config", None)
if not isinstance(condition_config, EdgeConditionConfig):
raw_value = payload.get("condition", "true")
condition_config = EdgeConditionConfig.from_dict(
raw_value,
path=extend_path(self.path, f"edge[{self.id}->{node.id}].condition"),
)
condition_label = condition_config.display_label()
condition_type = condition_config.type
condition_serializable = condition_config.to_external_value()
process_config = payload.pop("process_config", None)
if process_config is None and payload.get("process") is not None:
process_config = EdgeProcessorConfig.from_dict(
payload.get("process"),
path=extend_path(self.path, f"edge[{self.id}->{node.id}].process"),
)
process_serializable = process_config.to_external_value() if isinstance(process_config, EdgeProcessorConfig) else None
process_type = process_config.type if isinstance(process_config, EdgeProcessorConfig) else None
process_label = process_config.display_label() if isinstance(process_config, EdgeProcessorConfig) else None
# Handle dynamic_config
dynamic_config = payload.pop("dynamic_config", None)
if dynamic_config is None and payload.get("dynamic") is not None:
dynamic_config = DynamicEdgeConfig.from_dict(
payload.get("dynamic"),
path=extend_path(self.path, f"edge[{self.id}->{node.id}].dynamic"),
)
payload["condition"] = condition_serializable
payload["condition_label"] = condition_label
payload["condition_type"] = condition_type
if process_serializable is not None:
payload["process"] = process_serializable
payload["process_label"] = process_label
payload["process_type"] = process_type
if existing:
existing.config.update(payload)
existing.trigger = trigger
existing.condition = condition_label
existing.condition_config = condition_config
existing.condition_type = condition_type
existing.carry_data = carry_data
existing.keep_message = keep_message
existing.clear_context = clear_context
existing.clear_kept_context = clear_kept_context
if isinstance(process_config, EdgeProcessorConfig):
existing.process_config = process_config
existing.process_type = process_type
else:
existing.process_config = None
existing.process_type = None
existing.dynamic_config = dynamic_config
else:
self._outgoing_edges.append(
EdgeLink(
target=node,
config=payload,
trigger=trigger,
condition=condition_label,
condition_config=condition_config,
condition_type=condition_type,
carry_data=carry_data,
keep_message=keep_message,
clear_context=clear_context,
clear_kept_context=clear_kept_context,
process_config=process_config if isinstance(process_config, EdgeProcessorConfig) else None,
process_type=process_type,
dynamic_config=dynamic_config,
)
)
def add_predecessor(self, node: "Node") -> None:
if node not in self.predecessors:
self.predecessors.append(node)
def iter_outgoing_edges(self) -> Iterable[EdgeLink]:
return tuple(self._outgoing_edges)
def find_outgoing_edge(self, node_id: str) -> EdgeLink | None:
for link in self._outgoing_edges:
if link.target.id == node_id:
return link
return None
def is_triggered(self) -> bool:
if self.start_triggered:
return True
for predecessor in self.predecessors:
for edge_link in predecessor.iter_outgoing_edges():
if edge_link.target is self and edge_link.trigger and edge_link.triggered:
return True
return False
def reset_triggers(self) -> None:
self.start_triggered = False
for predecessor in self.predecessors:
for edge_link in predecessor.iter_outgoing_edges():
if edge_link.target is self:
edge_link.triggered = False
def merge_vars(self, parent_vars: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
merged = dict(parent_vars or {})
merged.update(self.vars)
return merged

32
entity/configs/node/passthrough.py Executable file
View File

@ -0,0 +1,32 @@
"""Configuration for passthrough nodes."""
from dataclasses import dataclass
from typing import Mapping, Any
from entity.configs.base import BaseConfig, ConfigFieldSpec, optional_bool, require_mapping
@dataclass
class PassthroughConfig(BaseConfig):
"""Configuration for passthrough nodes."""
only_last_message: bool = True
@classmethod
def from_dict(cls, data: Mapping[str, Any] | None, *, path: str) -> "PassthroughConfig":
if data is None:
return cls(only_last_message=True, path=path)
mapping = require_mapping(data, path)
only_last_message = optional_bool(mapping, "only_last_message", path, default=True)
return cls(only_last_message=only_last_message, path=path)
FIELD_SPECS = {
"only_last_message": ConfigFieldSpec(
name="only_last_message",
display_name="Only Last Message",
type_hint="bool",
required=False,
default=True,
description="If True, only pass the last received message. If False, pass all messages.",
),
}

97
entity/configs/node/python_runner.py Executable file
View File

@ -0,0 +1,97 @@
"""Configuration for Python code execution nodes."""
import sys
from dataclasses import dataclass, field
from typing import Any, Dict, List, Mapping
from entity.configs.base import (
BaseConfig,
ConfigError,
ConfigFieldSpec,
ensure_list,
optional_dict,
optional_str,
require_mapping,
)
def _default_interpreter() -> str:
return sys.executable or "python3"
@dataclass
class PythonRunnerConfig(BaseConfig):
interpreter: str = field(default_factory=_default_interpreter)
args: List[str] = field(default_factory=list)
env: Dict[str, str] = field(default_factory=dict)
timeout_seconds: int = 60
encoding: str = "utf-8"
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "PythonRunnerConfig":
mapping = require_mapping(data, path)
interpreter = optional_str(mapping, "interpreter", path) or _default_interpreter()
args_raw = mapping.get("args")
args = [str(item) for item in ensure_list(args_raw)] if args_raw is not None else []
env = optional_dict(mapping, "env", path) or {}
timeout_value = mapping.get("timeout_seconds", 60)
if not isinstance(timeout_value, int) or timeout_value <= 0:
raise ConfigError("timeout_seconds must be a positive integer", f"{path}.timeout_seconds")
encoding = optional_str(mapping, "encoding", path) or "utf-8"
if not encoding:
raise ConfigError("encoding cannot be empty", f"{path}.encoding")
return cls(
interpreter=interpreter,
args=args,
env={str(key): str(value) for key, value in env.items()},
timeout_seconds=timeout_value,
encoding=encoding,
path=path,
)
FIELD_SPECS = {
"interpreter": ConfigFieldSpec(
name="interpreter",
display_name="Python Path",
type_hint="str",
required=False,
default=_default_interpreter(),
description="Python executable file path, defaults to current process interpreter",
advance=True,
),
"args": ConfigFieldSpec(
name="args",
display_name="Startup Parameters",
type_hint="list[str]",
required=False,
default=[],
description="Parameter list appended after interpreter",
advance=True,
),
"env": ConfigFieldSpec(
name="env",
display_name="Additional Environment Variables",
type_hint="dict[str, str]",
required=False,
default={},
description="Additional environment variables, will override process defaults",
advance=True,
),
"timeout_seconds": ConfigFieldSpec(
name="timeout_seconds",
display_name="Timeout (seconds)",
type_hint="int",
required=False,
default=60,
description="Script execution timeout (seconds)",
),
"encoding": ConfigFieldSpec(
name="encoding",
display_name="Output Encoding",
type_hint="str",
required=False,
default="utf-8",
description="Encoding used to parse stdout/stderr",
advance=True,
),
}

266
entity/configs/node/subgraph.py Executable file
View File

@ -0,0 +1,266 @@
"""Subgraph node configuration and registry helpers."""
from dataclasses import dataclass, replace
from typing import Any, Dict, Mapping
from entity.enums import LogLevel
from entity.enum_options import enum_options_for, enum_options_from_values
from entity.configs.base import (
BaseConfig,
ConfigError,
ConfigFieldSpec,
ChildKey,
require_mapping,
require_str,
extend_path,
)
from entity.configs.edge.edge import EdgeConfig
from entity.configs.node.memory import MemoryStoreConfig
from utils.registry import Registry, RegistryError
subgraph_source_registry = Registry("subgraph_source")
def register_subgraph_source(
name: str,
*,
config_cls: type[BaseConfig],
description: str | None = None,
) -> None:
"""Register a subgraph source configuration class."""
metadata = {"summary": description} if description else None
subgraph_source_registry.register(name, target=config_cls, metadata=metadata)
def get_subgraph_source_config(name: str) -> type[BaseConfig]:
entry = subgraph_source_registry.get(name)
config_cls = entry.load()
if not isinstance(config_cls, type) or not issubclass(config_cls, BaseConfig):
raise RegistryError(f"Entry '{name}' is not a BaseConfig subclass")
return config_cls
def iter_subgraph_source_registrations() -> Dict[str, type[BaseConfig]]:
return {name: entry.load() for name, entry in subgraph_source_registry.items()}
def iter_subgraph_source_metadata() -> Dict[str, Dict[str, Any]]:
return {name: dict(entry.metadata or {}) for name, entry in subgraph_source_registry.items()}
@dataclass
class SubgraphFileConfig(BaseConfig):
file_path: str
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "SubgraphFileConfig":
mapping = require_mapping(data, path)
file_path = require_str(mapping, "path", path)
return cls(file_path=file_path, path=path)
FIELD_SPECS = {
"path": ConfigFieldSpec(
name="path",
display_name="Subgraph File Path",
type_hint="str",
required=True,
description="Subgraph file path (relative to yaml_instance/ or absolute path)",
),
}
@dataclass
class SubgraphInlineConfig(BaseConfig):
graph: Dict[str, Any]
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "SubgraphInlineConfig":
mapping = require_mapping(data, path)
return cls(graph=dict(mapping), path=path)
def validate(self) -> None:
if "nodes" not in self.graph:
raise ConfigError("subgraph config must define nodes", extend_path(self.path, "nodes"))
if "edges" not in self.graph:
raise ConfigError("subgraph config must define edges", extend_path(self.path, "edges"))
FIELD_SPECS = {
"id": ConfigFieldSpec(
name="id",
display_name="Subgraph ID",
type_hint="str",
required=True,
description="Subgraph identifier",
),
"description": ConfigFieldSpec(
name="description",
display_name="Subgraph Description",
type_hint="str",
required=False,
description="Describe the subgraph's responsibility, trigger conditions, and success criteria so reviewers know when to call it.",
),
"log_level": ConfigFieldSpec(
name="log_level",
display_name="Log Level",
type_hint="enum:LogLevel",
required=False,
default=LogLevel.INFO.value,
enum=[lvl.value for lvl in LogLevel],
description="Subgraph runtime log level",
enum_options=enum_options_for(LogLevel),
),
"is_majority_voting": ConfigFieldSpec(
name="is_majority_voting",
display_name="Majority Voting",
type_hint="bool",
required=False,
default=False,
description="Whether to perform majority voting on node results",
),
"nodes": ConfigFieldSpec(
name="nodes",
display_name="Node List",
type_hint="list[Node]",
required=True,
description="Subgraph node list, must contain at least one node",
),
"edges": ConfigFieldSpec(
name="edges",
display_name="Edge List",
type_hint="list[EdgeConfig]",
required=True,
description="Subgraph edge list",
child=EdgeConfig,
),
"memory": ConfigFieldSpec(
name="memory",
display_name="Memory Stores",
type_hint="list[MemoryStoreConfig]",
required=False,
description="Provide a list of memory stores if this subgraph needs dedicated stores; leave empty to inherit parent graph stores.",
child=MemoryStoreConfig,
),
"vars": ConfigFieldSpec(
name="vars",
display_name="Variables",
type_hint="dict[str, Any]",
required=False,
default={},
description="Variables passed to subgraph nodes",
),
"organization": ConfigFieldSpec(
name="organization",
display_name="Organization",
type_hint="str",
required=False,
description="Subgraph organization/team identifier",
),
"initial_instruction": ConfigFieldSpec(
name="initial_instruction",
display_name="Initial Instruction",
type_hint="str",
required=False,
description="Subgraph level initial instruction",
),
"start": ConfigFieldSpec(
name="start",
display_name="Start Node",
type_hint="str | list[str]",
required=False,
description="Start node ID list (entry list executed at subgraph start; not recommended to edit manually)",
),
"end": ConfigFieldSpec(
name="end",
display_name="End Node",
type_hint="str | list[str]",
required=False,
description="End node ID list (used to collect final subgraph output, not part of execution logic). This is an ordered list: earlier nodes are checked first; the first with output becomes the subgraph output, otherwise continue down the list.",
),
}
@classmethod
def field_specs(cls) -> Dict[str, ConfigFieldSpec]:
specs = super().field_specs()
nodes_spec = specs.get("nodes")
if nodes_spec:
from entity.configs.node.node import Node
specs["nodes"] = replace(nodes_spec, child=Node)
return specs
@dataclass
class SubgraphConfig(BaseConfig):
type: str
config: BaseConfig | None = None
@classmethod
def from_dict(cls, data: Mapping[str, Any], *, path: str) -> "SubgraphConfig":
mapping = require_mapping(data, path)
source_type = require_str(mapping, "type", path)
if "vars" in mapping and mapping["vars"]:
raise ConfigError("vars is only allowed at root level (DesignConfig.vars)", extend_path(path, "vars"))
if "config" not in mapping or mapping["config"] is None:
raise ConfigError("subgraph configuration requires 'config' block", extend_path(path, "config"))
try:
config_cls = get_subgraph_source_config(source_type)
except RegistryError as exc:
raise ConfigError(
f"subgraph.type must be one of {list(iter_subgraph_source_registrations().keys())}",
extend_path(path, "type"),
) from exc
config_obj = config_cls.from_dict(mapping["config"], path=extend_path(path, "config"))
return cls(type=source_type, config=config_obj, path=path)
def validate(self) -> None:
if not self.config:
raise ConfigError("subgraph config missing", extend_path(self.path, "config"))
if hasattr(self.config, "validate"):
self.config.validate()
FIELD_SPECS = {
"type": ConfigFieldSpec(
name="type",
display_name="Subgraph Source Type",
type_hint="str",
required=True,
description="Registered subgraph source such as 'config' or 'file' (see subgraph_source_registry).",
),
"config": ConfigFieldSpec(
name="config",
display_name="Subgraph Configuration",
type_hint="object",
required=True,
description="Payload interpreted by the chosen type—for example inline graph schema for 'config' or file path payload for 'file'.",
),
}
@classmethod
def child_routes(cls) -> Dict[ChildKey, type[BaseConfig]]:
return {
ChildKey(field="config", value=name): config_cls
for name, config_cls in iter_subgraph_source_registrations().items()
}
@classmethod
def field_specs(cls) -> Dict[str, ConfigFieldSpec]:
specs = super().field_specs()
type_spec = specs.get("type")
if type_spec:
registrations = iter_subgraph_source_registrations()
metadata = iter_subgraph_source_metadata()
names = list(registrations.keys())
descriptions = {
name: (metadata.get(name) or {}).get("summary") for name in names
}
specs["type"] = replace(
type_spec,
enum=names,
enum_options=enum_options_from_values(names, descriptions, preserve_label_case=True),
)
return specs

Some files were not shown because too many files have changed in this diff Show More