8.1 KiB
Models and services
Run these commands from the project directory (ChatDev/execorchestra) after installing the host environment
and running python -m scripts.prepare_release init. Keep all service credentials
in .env, and export them with set -a; source .env; set +a in each shell.
Use shell-compatible values; quote values containing spaces or shell characters.
Model endpoint
Set LLM_BASE_URL to the OpenAI-compatible API base URL and LLM_API_KEY to
its credential. The endpoint must support strict JSON-schema responses,
configured reasoning settings, tool continuation, and usage metadata.
| Control role | API model ID | Reasoning |
|---|---|---|
| Planner | gpt-6.1-sol |
high |
| Profiler | gemini-3.8-flash |
high |
| Composer and plan adaptation | gpt-6-luna |
high |
| Optional evaluator | gpt-5.6-sol |
high |
The optional evaluator is disabled in the supplied execution configuration.
gpt-6.1-sol and gpt-6-luna are registered for control roles; adding them
does not add them to the task candidate pool. The distributed research pool contains 10 models, 38 tools, 4 skills, and 5 agents. The candidate model API IDs are:
deepseek-v4.1-flash
deepseek-v4-pro-0813
gemini-3.8-flash
glm-5.3
gpt-5.6-luna
gpt-5.6-sol
gpt-6-astra
qwen3.5-35b-a3b
qwen3.8-flash
qwen3-coder-next
Health probes cover the model registry in model_selection.json; readiness
requires the configured candidate pool. Map API identifiers to the corresponding
model versions.
Embedding service
The reference deployment uses Qwen/Qwen3-Embedding-4B, revision
5cf2132abc99cad020ac570b19d031efec650f2b, bfloat16 inference, and 2,560-dimensional
vectors.
Download the model and tokenizer
For a local deployment, prepare its separate GPU environment and download the weights and matching tokenizer. These commands do not start the service:
conda create -n execorchestra-embedding python=3.11 -y
conda run -n execorchestra-embedding python -m pip install -r requirements-embedding.txt
python -m scripts.download_embedding
For an existing remote service, download only the matching tokenizer. No GPU environment or weights are needed on the framework host:
python -m scripts.download_embedding --tokenizer-only
Start the local service
After configuration, open a separate terminal at the project directory (ChatDev/execorchestra). Set a
private, nonempty SGAR_EMBEDDING_API_KEY in .env and use an NVIDIA GPU
supporting BF16. On a dedicated GPU server, start the prepared service with:
conda activate execorchestra-embedding
set -a
source .env
set +a
export SGAR_EMBEDDING_MODEL_PATH="$PWD/.cache/embedding-model"
# On a dedicated GPU server only; Slurm users instead run inside an allocation.
export SGAR_EMBEDDING_DEDICATED_GPU=1
python -m scripts.embedding_server
The service binds to 127.0.0.1:8001. It uses BF16, SDPA, last-token pooling,
L2 normalization, and a fixed microbatch of one. It does not silently truncate
inputs or add a second retrieval instruction. Stop the foreground process after
the task; the module does not start automatically during installation or checks.
The model directory must contain the weights and tokenizer for the pinned revision.
An NVIDIA driver and NVIDIA Container Toolkit are required for the alternative vLLM container deployment:
export EMBEDDING_GPU=0
VLLM_API_KEY="$SGAR_EMBEDDING_API_KEY" docker run -d \
--name execorchestra-embedding --gpus "device=${EMBEDDING_GPU}" \
--network host --ipc host -e VLLM_API_KEY \
-v "$PWD/.cache/embedding-model:/models/embedding-4b:ro" \
vllm/vllm-openai:v0.27.1-cu129-ubuntu2404 \
/models/embedding-4b --served-model-name sgar-embedding-4b \
--host 127.0.0.1 --port 8001 --dtype bfloat16 \
--gpu-memory-utilization 0.80 --max-model-len 32768
The client template points to
http://127.0.0.1:8001/v1/embeddings with API model ID sgar-embedding-4b.
For a remote service, update runtime.api_endpoint_url and, if necessary,
runtime.api_model_id in sgar_mvp/config/embedding_release.json. Keep the
model revision, dimension, instructions, and normalization settings unchanged.
The host client adds retrieval instructions. Size GPU memory for the configured
sequence length and service concurrency.
The framework sends {"model": "sgar-embedding-4b", "input": ["..."]} and expects
data entries with unique zero-based index values and 2,560-element embedding
arrays. Export the API key even for a local service.
Tool services
The complete candidate list includes the three services below. These endpoints
must be reachable from tool containers; 127.0.0.1 inside a container does not
refer to the host.
| Environment variable | Required interface |
|---|---|
SEARXNG_BASE_URL |
SearXNG root URL, with JSON search enabled at /search. |
ZOTERO_TRANSLATION_SERVER_URL |
Translation Server root URL, accepting text identifiers at /search. |
HYPOTHESIS_API_URL |
Hypothesis API root, including /api, accepting annotation creation at /annotations. |
HYPOTHESIS_API_TOKEN |
Write-capable token for an isolated Hypothesis instance. |
For SearXNG and Zotero, the following starts separate services on the default Docker bridge gateway. Use an administrator-approved bind address if your host uses different networking:
export DOCKER_GATEWAY="$(docker network inspect bridge -f '{{(index .IPAM.Config 0).Gateway}}')"
mkdir -p .cache/services/searxng
python - <<'PY'
from pathlib import Path
import secrets
p = Path(".cache/services/searxng/settings.yml")
if not p.exists():
p.write_text("use_default_settings: true\nsearch:\n formats: [html, json]\n"
"server:\n limiter: false\n secret_key: " + secrets.token_hex(32) + "\n")
PY
docker run -d --name execorchestra-search \
-p "${DOCKER_GATEWAY}:8888:8080" \
-v "$PWD/.cache/services/searxng:/etc/searxng" \
searxng/searxng@sha256:394bc18556af80cd1cd448daf5b69ead7f65d990912851a47ba1ec79c85fec44
docker run -d --name execorchestra-translation \
-p "${DOCKER_GATEWAY}:1969:1969" zotero/translation-server:2.0.3
Set SEARXNG_BASE_URL=http://<bridge-gateway>:8888 and
ZOTERO_TRANSLATION_SERVER_URL=http://<bridge-gateway>:1969 in .env, replacing
<bridge-gateway> with the value above. Include that address in proxy bypass
lists. SearXNG also needs outbound access to its configured search engines.
See the SearXNG deployment documentation
and Zotero Translation Server
for service administration.
Use an isolated Hypothesis server
and its token for annotation tools. The retained environment used server version
v0.39.0. Its upstream deployment includes a database, search backend, and worker;
those services and their state are not part of the framework repository.
The tool check creates an annotation; use a private test instance. This service
is required by the complete candidate pool.
Network configuration
Framework requests use explicit routes:
| Setting | Applies to |
|---|---|
SGAR_MODEL_PROXY |
Model API traffic and the optional cold-image downloader. Leave empty only when these routes work directly or images are pre-pulled. |
SGAR_TOOL_HTTP_PROXY, SGAR_TOOL_HTTPS_PROXY |
Network-enabled tool containers. |
SGAR_TOOL_NO_PROXY |
Local/internal tool-service addresses bypassing those proxies. |
SGAR_DOCKER_PROXY |
TB task containers; use an HTTP CONNECT proxy reachable from the Docker bridge. |
SGAR_NO_PROXY |
TB container proxy exclusions. |
HTTP_PROXY, HTTPS_PROXY, NO_PROXY |
Host downloads, image builds, and preparation tool subprocesses. |
Use container-reachable proxy addresses for tools and TB. Configure
SGAR_DOCKER_PROXY explicitly: the adapter's default is a bridge-gateway relay
on port 7894. Model clients use SGAR_MODEL_PROXY; embedding requests to loopback
or private IP literals bypass it.
Export .env before preparation and execution. Public-network tools require
--network-policy declared in the framework/GAIA launcher or --network in the
TB launcher; these flags do not override a task's own network restrictions.