ChatDev/execorchestra/docs/XBENCH_DEMO.md

1.8 KiB

XBench web research example

XBench DeepSearch-2510 case 157 exercises search, page reading, and answer delivery. Run this command from ChatDev/execorchestra once the framework environment, Docker runtime image, model gateway, and embedding service are ready. The command handles the resource preparation needed for this example. Quick Start

bash scripts/run_xbench_demo.sh

The command loads the project's .env, prepares the case locally from the pinned dataset, and starts temporary SearXNG and Chromium services on the local Docker bridge. It checks for search results and browser readiness before starting the framework. The model gateway and embedding endpoint come from the Quick Start setup. The example supplies its own search and browser addresses and browser token. Pinned dataset

If the prepared index already includes both web tools, the example reuses it. Otherwise, it prepares a separate checkout of the current commit and builds the index there. This step makes live API calls and may incur charges. The first run downloads or builds the service images. Sites and search engines must be reachable from the Docker host. On networks that require a tool proxy, use the same optional proxy settings as in the framework. Network settings

The command prints a new directory under runs/xbench-157-*/. It contains the service logs and, when applicable, the preparation report. Open run/progress.log for the trace, run/run_manifest.json for completion, and run/delivery_manifest.json for the delivered file location. The temporary service containers stop when the command exits. Keep the decoded question and its run outputs local. Benchmark grading is a separate step.