mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 13:16:24 +00:00
fix(integrations): keep Lark runtime status consistent (#5994)
* fix(integrations): keep Lark runtime status consistent Probe sandbox runtime readiness for every Lark mutation response outside the credential lock. Expose explicit sandbox_runtime_probed state so the frontend can distinguish conservative fallbacks without inferring semantics from runtime detail text, while cancelling stale reads before updating the cache. * style(frontend): format Lark cache test * fix(integrations): snapshot Lark credential status under the lock Why this change is needed: PR #5994 moved the post-mutation status read outside the per-user credential lock so the new sandbox runtime probe would not extend the critical section. As flagged in review, that lets a concurrent app switch or auth flow land between the committed mutation and the status read; in remote-provisioner mode the runtime probe can spend seconds before probe_lark_auth runs, so the raced switch can clear the new tokens and make a completed login report success=false, or mix an old app_id with a newer auth state. What changed: - add _LarkCredentialSnapshot (app config + auth probe) and capture it in the same _lark_credential_lock section that commits the mutation - thread the snapshot through _get_lark_mutation_status into get_lark_integration_status(credential_snapshot=...); only the credential-independent runtime probe still runs after the lock - apply to complete_lark_auth, complete_lark_config, and set_lark_app_credentials; install keeps the one-shot status build Problem solved: Mutation responses stay tied to the just-committed credential generation: a raced app switch can no longer flip a completed device-code login to success=false or blend a stale app_id with newer auth state, while the runtime probe stays off the lock. * fix(integrations): address Lark status review follow-ups - install_lark_integration: snapshot credential-derived status fields under _lark_credential_lock so the install response cannot mix newer credential state read across the unlocked runtime probe - useInstallLarkIntegration: route the cache write through cacheLarkMutationStatus so install cancels in-flight reads and keeps probed runtime fields under frontend/backend version skew - README: soften the UI claim - the status API exposes sandbox_runtime_probed and the mutation cache honors it * test(integrations): tighten Lark status fixtures and guidance docs - cache.test: use a distinct mutation runtime mode in the unprobed-merge case so the assertions prove the cached runtime fields win - e2e mock: include broker in the sandbox_runtime_mode union cast so the fixture cannot drift from LarkSandboxRuntimeMode - AGENTS.md (gateway + skills): mention sandbox_runtime_probed alongside the existing runtime readiness fields, keeping the gateway guidance file under its byte budget * fix(integrations): harden Lark mutation cache against skew and races - cacheLarkMutationStatus: await cancelQueries before writing so an in-flight GET whose fetch already resolved cannot clobber the fresher mutation status, matching the existing threads/archive pattern - self-heal version skew: an unprobed (pre-field backend) mutation response now triggers a status refetch instead of risking the conservative runtime fallback lingering until the next focus/remount - api: normalize a missing sandbox_runtime_probed to false at the boundary so the declared-non-optional type matches skewed runtimes - tests: pin the skew contract (missing-field cases for cache and api) and add a useInstallLarkIntegration regression test proving the install cache write goes through cacheLarkMutationStatus - README: describe the probed flag as cross-version semantics instead of claiming the same-version API surfaces both states - lark_cli: note that install's credential snapshot runs the auth probe subprocess while holding the credential lock * refactor(integrations): derive sandbox_runtime_probed inside the resolver _resolve_sandbox_runtime_readiness now returns a fourth probed element next to the branch that produces the unevaluated remote fallback, so the 'was readiness actually evaluated' invariant can no longer drift from the fallback logic it mirrors. Callers and resolver-level tests updated to pin probed per branch.
This commit is contained in:
parent
907910957b
commit
1b691689da
@ -1170,7 +1170,11 @@ no hostPath/PVC runtime mount. Publish the image under
|
||||
[`docker/lark-cli-init`](docker/lark-cli-init/README.md) and set
|
||||
`LARK_CLI_INIT_IMAGE` on the provisioner; it stays off (legacy behavior) when
|
||||
unset. The Lark integration status (`GET /api/integrations/lark/status`) reports
|
||||
`sandbox_runtime_mode` and `sandbox_runtime_ready` so the Settings UI shows
|
||||
`sandbox_runtime_mode`, `sandbox_runtime_probed`, and `sandbox_runtime_ready`.
|
||||
`sandbox_runtime_probed` marks whether runtime readiness was actually
|
||||
evaluated; responses from older backends may omit the flag, in which case the
|
||||
Settings mutation cache keeps the last probed runtime fields instead of
|
||||
overwriting them with an unevaluated fallback — so the Settings UI shows
|
||||
whether `lark-cli` will actually be present in the sandbox at chat time, rather
|
||||
than a green status hiding a later `command not found`.
|
||||
|
||||
|
||||
@ -112,7 +112,7 @@ owner-scoped assistant version selection remains enabled.
|
||||
| **MCP Tasks** (`/api/threads/{id}/mcp-tasks`) | `GET /` - current user's durable tasks for one owned thread; `GET /{task_id}` - bounded result/input/status-error/cancellation-error detail, including cancellation attempt count, without remote task IDs or driver configuration |
|
||||
| **Skills** (`/api/skills`) | `GET /` - list; `GET /{name}` - inspect; `PUT /{name}` - toggle; `POST /install` - install a thread-local .skill archive; `POST /install/upload` - admin-only multipart, authorized before parsing and capped at a 100 MiB file plus 1 MiB framing; `POST /reload` - invalidate process-local cache after trusted filesystem changes |
|
||||
| **Subagents** (`/api/subagents`) | Admin managed-worker CRUD and listing. |
|
||||
| **Integrations** (`/api/integrations`) | `GET /lark/status` - inspect managed Lark/Feishu CLI integration state, including `sandbox_runtime_mode` / `sandbox_runtime_ready` (whether `lark-cli` will actually be present in the sandbox at chat time); `POST /lark/install` - admin-only install of the official `lark-*` managed skill pack; `POST /lark/config/start` and `/lark/config/complete` - internal first-time Lark connection setup; `POST /lark/config/credentials` - atomically switch the caller's per-user Lark app after validating the new `app_id`/`app_secret` through the official CLI's live tenant-token probe, revoke/remove the previous OAuth tokens, and restore the prior credential tree if the switch fails; `POST /lark/auth/start` and `/lark/auth/complete` - browser device-flow user authorization without terminal access, with optional `domains` / exact `scope` for incremental permission grants. Config and auth flows carry a server-issued, per-user generation persisted under the credential lock; a rejected direct switch leaves the current generation unchanged, stale completions return 409, and browser re-registration uses the same token-clearing/revocation transaction as direct credential switches. |
|
||||
| **Integrations** (`/api/integrations`) | `GET /lark/status` - inspect managed Lark/Feishu CLI integration state, including `sandbox_runtime_mode` / `sandbox_runtime_probed` / `sandbox_runtime_ready` (whether `lark-cli` will be present in the sandbox at chat time); `POST /lark/install` - admin-only install of the official `lark-*` managed skill pack; `POST /lark/config/start` and `/lark/config/complete` - internal first-time Lark connection setup; `POST /lark/config/credentials` - atomically switch the caller's per-user Lark app after validating the new `app_id`/`app_secret` through the official CLI's live tenant-token probe, revoke/remove the previous OAuth tokens, and restore the prior credential tree if the switch fails; `POST /lark/auth/start` and `/lark/auth/complete` - browser device-flow user authorization without terminal access, with optional `domains` / exact `scope` for incremental permission grants. Config and auth flows carry a server-issued, per-user generation persisted under the credential lock; a rejected direct switch leaves the current generation unchanged, stale completions return 409, and browser re-registration uses the same token-clearing/revocation transaction as direct credential switches. |
|
||||
| **Memory** (`/api/memory`) | `GET /` - memory data; `POST /reload` - force reload; `GET /config` - config; `GET /status` - config + data |
|
||||
| **Uploads** (`/api/threads/{id}/uploads`) | `POST /` - upload files (auto-converts PDF/PPT/Excel/Word); non-mounted sync releases its lease after the last holder exits; `GET /list` - list; `DELETE /{filename}` - delete a regular file; symlinks 404 and a converted `.md` is kept (#5672) |
|
||||
| **Threads** (`/api/threads/{id}`) | `DELETE /` - remove DeerFlow-managed local thread data after LangGraph thread deletion; `POST /branches` - branch a completed assistant turn with a replay checkpoint; inherited titles take next-free displayed sibling suffixes, including explicit/renamed ones, while explicit titles stay unchanged. Durable `branch` admission rejects races. Workspace files are not checkpointed, so the branch only best-effort copies the current workspace when branching from the **latest** turn (`workspace_clone_mode="current_thread_best_effort"`); branching from an older/historical turn skips the copy (`workspace_clone_mode="skipped_historical_turn"`) so the branch never inherits files that only exist in a later timeline. Thread-scoped runtime channels (`sandbox`, `thread_data`) are not copied onto the branch: the parent's `sandbox_id` binds path mappings and the release lifecycle to the parent's workspace, so the branch lazily acquires its own sandbox instead. Branch creation also seeds the new thread's run-event feed from the branch checkpoint's visible messages (`history_seed_mode` in the response): the thread feed reads run_events, not checkpoints, so without the seed the inherited history disappears from the UI after the branch's first run (#4380). Seeded rows are grouped into one synthetic run per inherited turn (`branch-seed-{thread_id}-{n}`, a new turn opening at every persisted human message, including an allowlisted hidden `ask_clarification` reply) because `run_id` is a turn identity to the feed's consumers, not a provenance tag: regenerating an inherited answer supersedes that row's whole `run_id` in `GET /messages/page`, so one shared id for the entire seed deleted the complete inherited history on a branch's first regenerate (#4458); `GET /goal`, `PUT /goal`, `DELETE /goal` - read, set, and clear the active thread goal; `POST /compact` - summarize older active context, deriving memory policy and bucket from the state-producing checkpoint rather than request `agent_name`, and block while a run is in flight; unexpected failures return a generic 500 detail |
|
||||
|
||||
@ -82,6 +82,7 @@ class LarkIntegrationStatusResponse(BaseModel):
|
||||
cli: LarkCliProbeResponse
|
||||
auth: LarkAuthProbeResponse
|
||||
sandbox_runtime_mode: str = Field("none", description="How lark-cli is provisioned into the sandbox: none, gateway-download, init-container, or broker")
|
||||
sandbox_runtime_probed: bool = Field(False, description="Whether sandbox runtime readiness was evaluated rather than conservatively defaulted")
|
||||
sandbox_runtime_ready: bool = Field(False, description="Whether the sandbox lark-cli runtime is provisioned and usable at chat time")
|
||||
sandbox_runtime_detail: str | None = Field(None, description="Human-readable reason when the sandbox runtime is not ready")
|
||||
|
||||
@ -202,6 +203,7 @@ def _status_to_response(status: LarkIntegrationStatus, *, include_host_paths: bo
|
||||
cli=cli,
|
||||
auth=_auth_probe_to_response(status.auth),
|
||||
sandbox_runtime_mode=status.sandbox_runtime_mode,
|
||||
sandbox_runtime_probed=status.sandbox_runtime_probed,
|
||||
sandbox_runtime_ready=status.sandbox_runtime_ready,
|
||||
sandbox_runtime_detail=status.sandbox_runtime_detail,
|
||||
)
|
||||
|
||||
@ -215,6 +215,7 @@ class LarkIntegrationStatus:
|
||||
cli: LarkCliProbe
|
||||
auth: LarkAuthProbe
|
||||
sandbox_runtime_mode: str = "none"
|
||||
sandbox_runtime_probed: bool = False
|
||||
sandbox_runtime_ready: bool = False
|
||||
sandbox_runtime_detail: str | None = None
|
||||
|
||||
@ -1616,7 +1617,7 @@ def _resolve_sandbox_runtime_readiness(
|
||||
config: AppConfig,
|
||||
*,
|
||||
probe: bool,
|
||||
) -> tuple[str, bool, str | None]:
|
||||
) -> tuple[str, bool, str | None, bool]:
|
||||
"""Resolve the sandbox lark-cli runtime mode and readiness.
|
||||
|
||||
Modes:
|
||||
@ -1632,31 +1633,37 @@ def _resolve_sandbox_runtime_readiness(
|
||||
supersedes ``init-container`` when both are available.
|
||||
|
||||
``probe`` gates the (best-effort, short-timeout) provisioner capability call.
|
||||
|
||||
The fourth element reports whether readiness was actually evaluated. It is
|
||||
False only for the unevaluated remote fallback (``probe=False`` with a
|
||||
remote provisioner) and lives next to that branch so the two can never
|
||||
drift apart; every other mode resolves readiness deterministically or via
|
||||
a real probe, so it reports True.
|
||||
"""
|
||||
if not _uses_aio_sandbox(config):
|
||||
return "none", False, "Sandbox does not run lark-cli in this configuration."
|
||||
return "none", False, "Sandbox does not run lark-cli in this configuration.", True
|
||||
|
||||
if _uses_remote_provisioner(config):
|
||||
if not probe:
|
||||
return "init-container", False, None
|
||||
return "init-container", False, None, False
|
||||
caps = _probe_provisioner_capabilities(config)
|
||||
if caps is None:
|
||||
return "init-container", False, "Could not reach the provisioner to confirm the lark-cli runtime image."
|
||||
return "init-container", False, "Could not reach the provisioner to confirm the lark-cli runtime image.", True
|
||||
# Pattern B (broker) supersedes Pattern A (init-container binary) when
|
||||
# the provisioner has a broker image configured.
|
||||
if caps["lark_cli_broker_image"]:
|
||||
return "broker", True, None
|
||||
return "broker", True, None, True
|
||||
if caps["lark_cli_init_image"]:
|
||||
return "init-container", True, None
|
||||
return "init-container", False, "The provisioner has no lark-cli runtime image configured (LARK_CLI_INIT_IMAGE / LARK_CLI_BROKER_IMAGE)."
|
||||
return "init-container", True, None, True
|
||||
return "init-container", False, "The provisioner has no lark-cli runtime image configured (LARK_CLI_INIT_IMAGE / LARK_CLI_BROKER_IMAGE).", True
|
||||
|
||||
# Local AIO: Gateway-download runtime dir.
|
||||
runtime_dir = lark_cli_managed_sandbox_dir()
|
||||
try:
|
||||
_validate_lark_cli_sandbox_runtime(runtime_dir)
|
||||
except (ValueError, OSError):
|
||||
return "gateway-download", False, "The managed sandbox lark-cli runtime is not installed."
|
||||
return "gateway-download", True, None
|
||||
return "gateway-download", False, "The managed sandbox lark-cli runtime is not installed.", True
|
||||
return "gateway-download", True, None, True
|
||||
|
||||
|
||||
LARK_BROKER_MODE_TTL_SECONDS = 60
|
||||
@ -1714,6 +1721,29 @@ def sandbox_lark_broker_active(config: AppConfig | None = None) -> bool:
|
||||
return active
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _LarkCredentialSnapshot:
|
||||
"""Credential-derived status fields captured under ``_lark_credential_lock``.
|
||||
|
||||
Mutation endpoints must snapshot these fields in the same critical section
|
||||
that commits the credential change; the slower, credential-independent
|
||||
runtime probe can then run after the lock is released without letting a
|
||||
concurrent app switch or auth flow mix newer credential state (or cleared
|
||||
tokens) into the response for the just-completed mutation.
|
||||
"""
|
||||
|
||||
app_config: dict[str, str | bool | None]
|
||||
auth: LarkAuthProbe
|
||||
|
||||
|
||||
def _read_lark_credential_snapshot(user_id: str, *, verify_auth: bool) -> _LarkCredentialSnapshot:
|
||||
"""Read the credential-derived status fields; call under the credential lock."""
|
||||
return _LarkCredentialSnapshot(
|
||||
app_config=read_lark_app_config(user_id),
|
||||
auth=probe_lark_auth(user_id, verify=verify_auth),
|
||||
)
|
||||
|
||||
|
||||
def get_lark_integration_status(
|
||||
user_id: str,
|
||||
config: AppConfig,
|
||||
@ -1721,16 +1751,19 @@ def get_lark_integration_status(
|
||||
verify_auth: bool = False,
|
||||
check_latest: bool = False,
|
||||
check_runtime: bool = False,
|
||||
credential_snapshot: _LarkCredentialSnapshot | None = None,
|
||||
) -> LarkIntegrationStatus:
|
||||
root = lark_integration_root(user_id)
|
||||
manifest = _read_manifest(root)
|
||||
app_config = read_lark_app_config(user_id)
|
||||
if credential_snapshot is None:
|
||||
credential_snapshot = _read_lark_credential_snapshot(user_id, verify_auth=verify_auth)
|
||||
app_config = credential_snapshot.app_config
|
||||
installed_skills = tuple(sorted(_installed_lark_skill_names(root)))
|
||||
enabled_skills = tuple(sorted(_enabled_lark_skill_names(user_id, config)))
|
||||
manifest_version = str(manifest.get("version")) if manifest else None
|
||||
cli = probe_lark_cli()
|
||||
latest_available = _cached_latest_lark_cli_version() if check_latest else None
|
||||
runtime_mode, runtime_ready, runtime_detail = _resolve_sandbox_runtime_readiness(config, probe=check_runtime)
|
||||
runtime_mode, runtime_ready, runtime_detail, runtime_probed = _resolve_sandbox_runtime_readiness(config, probe=check_runtime)
|
||||
return LarkIntegrationStatus(
|
||||
installed=bool(manifest) and "lark-shared" in installed_skills,
|
||||
version=manifest_version or FALLBACK_LARK_CLI_VERSION,
|
||||
@ -1746,13 +1779,39 @@ def get_lark_integration_status(
|
||||
enabled_skills=enabled_skills,
|
||||
install_path=str(root),
|
||||
cli=cli,
|
||||
auth=probe_lark_auth(user_id, verify=verify_auth),
|
||||
auth=credential_snapshot.auth,
|
||||
sandbox_runtime_mode=runtime_mode,
|
||||
sandbox_runtime_probed=runtime_probed,
|
||||
sandbox_runtime_ready=runtime_ready,
|
||||
sandbox_runtime_detail=runtime_detail,
|
||||
)
|
||||
|
||||
|
||||
def _get_lark_mutation_status(
|
||||
user_id: str,
|
||||
config: AppConfig,
|
||||
*,
|
||||
verify_auth: bool = False,
|
||||
credential_snapshot: _LarkCredentialSnapshot | None = None,
|
||||
) -> LarkIntegrationStatus:
|
||||
"""Build a mutation response with the same runtime probe used by the GET route.
|
||||
|
||||
Pass ``credential_snapshot`` (captured under ``_lark_credential_lock`` right
|
||||
after the mutation) so the credential-derived fields stay tied to the
|
||||
committed change while the independent runtime probe runs unlocked; without
|
||||
it, a concurrent app switch or auth flow landing during that probe could
|
||||
clear the new tokens or mix a newer ``app_id``/auth state into this
|
||||
response.
|
||||
"""
|
||||
return get_lark_integration_status(
|
||||
user_id,
|
||||
config,
|
||||
verify_auth=verify_auth,
|
||||
check_runtime=True,
|
||||
credential_snapshot=credential_snapshot,
|
||||
)
|
||||
|
||||
|
||||
def _normalize_version(value: str | None) -> str | None:
|
||||
"""Extract a comparable ``major.minor.patch`` from a version-ish string."""
|
||||
if not value:
|
||||
@ -1855,7 +1914,17 @@ def install_lark_integration(
|
||||
sandbox_version = str(installed_manifest.get("version") or resolved_version or FALLBACK_LARK_CLI_VERSION)
|
||||
_ensure_managed_sandbox_lark_cli(sandbox_version)
|
||||
|
||||
status = get_lark_integration_status(user_id, config)
|
||||
# Install does not change credentials, but it shares the mutation-status
|
||||
# discipline: snapshot the credential-derived fields under the credential
|
||||
# lock so a concurrent app switch or auth flow landing during the unlocked
|
||||
# runtime probe cannot mix newer credential state into this response. The
|
||||
# snapshot's auth probe runs `lark-cli auth status` as a subprocess
|
||||
# (worst case its 8s timeout), so a concurrent same-user mutation may now
|
||||
# wait on the lock for that duration — an accepted trade-off since install
|
||||
# itself never mutates credentials.
|
||||
with _lark_credential_lock(user_id):
|
||||
credential_snapshot = _read_lark_credential_snapshot(user_id, verify_auth=False)
|
||||
status = _get_lark_mutation_status(user_id, config, credential_snapshot=credential_snapshot)
|
||||
content_changed = previous_content_sha is not None and previous_content_sha != content_sha
|
||||
message = f"Installed {len(installed_skills)} Lark/Feishu skills."
|
||||
if content_changed:
|
||||
@ -1990,7 +2059,8 @@ def complete_lark_config(
|
||||
app_secret=app_secret,
|
||||
brand=final_brand,
|
||||
)
|
||||
status = get_lark_integration_status(user_id, config)
|
||||
credential_snapshot = _read_lark_credential_snapshot(user_id, verify_auth=False)
|
||||
status = _get_lark_mutation_status(user_id, config, credential_snapshot=credential_snapshot)
|
||||
return LarkConfigCompleteResult(
|
||||
success=True,
|
||||
status=status,
|
||||
@ -2027,7 +2097,8 @@ def set_lark_app_credentials(
|
||||
app_secret=app_secret,
|
||||
brand=parsed_brand,
|
||||
)
|
||||
status = get_lark_integration_status(user_id, config)
|
||||
credential_snapshot = _read_lark_credential_snapshot(user_id, verify_auth=False)
|
||||
status = _get_lark_mutation_status(user_id, config, credential_snapshot=credential_snapshot)
|
||||
return LarkConfigCompleteResult(
|
||||
success=True,
|
||||
status=status,
|
||||
@ -2105,7 +2176,8 @@ def complete_lark_auth(
|
||||
timeout=wait_timeout_seconds,
|
||||
allow_empty_success=True,
|
||||
)
|
||||
status = get_lark_integration_status(user_id, config, verify_auth=True)
|
||||
credential_snapshot = _read_lark_credential_snapshot(user_id, verify_auth=True)
|
||||
status = _get_lark_mutation_status(user_id, config, verify_auth=True, credential_snapshot=credential_snapshot)
|
||||
return LarkAuthCompleteResult(
|
||||
success=status.auth.status == "authenticated",
|
||||
status=status,
|
||||
|
||||
File diff suppressed because one or more lines are too long
@ -553,10 +553,11 @@ def test_remote_provisioner_install_skips_gateway_sandbox_runtime(monkeypatch, t
|
||||
def test_status_runtime_mode_none_for_non_aio(monkeypatch, tmp_path) -> None:
|
||||
config = _config(tmp_path / "skills")
|
||||
config.sandbox = SimpleNamespace(use="deerflow.sandbox.local:LocalSandboxProvider")
|
||||
mode, ready, detail = lark_cli._resolve_sandbox_runtime_readiness(config, probe=True)
|
||||
mode, ready, detail, probed = lark_cli._resolve_sandbox_runtime_readiness(config, probe=True)
|
||||
assert mode == "none"
|
||||
assert ready is False
|
||||
assert detail
|
||||
assert probed is True
|
||||
|
||||
|
||||
def test_status_runtime_mode_gateway_download_ready(monkeypatch, tmp_path) -> None:
|
||||
@ -575,20 +576,22 @@ def test_status_runtime_mode_gateway_download_ready(monkeypatch, tmp_path) -> No
|
||||
target.write_bytes(b"\x7fELF")
|
||||
target.chmod(0o755)
|
||||
|
||||
mode, ready, detail = lark_cli._resolve_sandbox_runtime_readiness(config, probe=True)
|
||||
mode, ready, detail, probed = lark_cli._resolve_sandbox_runtime_readiness(config, probe=True)
|
||||
assert mode == "gateway-download"
|
||||
assert ready is True
|
||||
assert detail is None
|
||||
assert probed is True
|
||||
|
||||
|
||||
def test_status_runtime_mode_gateway_download_not_ready(monkeypatch, tmp_path) -> None:
|
||||
_patch_paths(monkeypatch, tmp_path / "home")
|
||||
config = _config(tmp_path / "skills")
|
||||
config.sandbox = SimpleNamespace(use="deerflow.community.aio_sandbox:AioSandboxProvider")
|
||||
mode, ready, detail = lark_cli._resolve_sandbox_runtime_readiness(config, probe=True)
|
||||
mode, ready, detail, probed = lark_cli._resolve_sandbox_runtime_readiness(config, probe=True)
|
||||
assert mode == "gateway-download"
|
||||
assert ready is False
|
||||
assert detail
|
||||
assert probed is True
|
||||
|
||||
|
||||
def test_status_runtime_mode_init_container_ready(monkeypatch, tmp_path) -> None:
|
||||
@ -598,10 +601,11 @@ def test_status_runtime_mode_init_container_ready(monkeypatch, tmp_path) -> None
|
||||
provisioner_url="http://provisioner:8002",
|
||||
)
|
||||
monkeypatch.setattr(lark_cli, "_probe_provisioner_capabilities", lambda _config: {"lark_cli_init_image": True, "lark_cli_broker_image": False})
|
||||
mode, ready, detail = lark_cli._resolve_sandbox_runtime_readiness(config, probe=True)
|
||||
mode, ready, detail, probed = lark_cli._resolve_sandbox_runtime_readiness(config, probe=True)
|
||||
assert mode == "init-container"
|
||||
assert ready is True
|
||||
assert detail is None
|
||||
assert probed is True
|
||||
|
||||
|
||||
def test_status_runtime_mode_broker_supersedes_init_container(monkeypatch, tmp_path) -> None:
|
||||
@ -612,10 +616,11 @@ def test_status_runtime_mode_broker_supersedes_init_container(monkeypatch, tmp_p
|
||||
)
|
||||
# Broker (Pattern B) wins even when the init image is also configured.
|
||||
monkeypatch.setattr(lark_cli, "_probe_provisioner_capabilities", lambda _config: {"lark_cli_init_image": True, "lark_cli_broker_image": True})
|
||||
mode, ready, detail = lark_cli._resolve_sandbox_runtime_readiness(config, probe=True)
|
||||
mode, ready, detail, probed = lark_cli._resolve_sandbox_runtime_readiness(config, probe=True)
|
||||
assert mode == "broker"
|
||||
assert ready is True
|
||||
assert detail is None
|
||||
assert probed is True
|
||||
|
||||
|
||||
def test_status_runtime_mode_init_container_not_configured(monkeypatch, tmp_path) -> None:
|
||||
@ -625,10 +630,11 @@ def test_status_runtime_mode_init_container_not_configured(monkeypatch, tmp_path
|
||||
provisioner_url="http://provisioner:8002",
|
||||
)
|
||||
monkeypatch.setattr(lark_cli, "_probe_provisioner_capabilities", lambda _config: {"lark_cli_init_image": False, "lark_cli_broker_image": False})
|
||||
mode, ready, detail = lark_cli._resolve_sandbox_runtime_readiness(config, probe=True)
|
||||
mode, ready, detail, probed = lark_cli._resolve_sandbox_runtime_readiness(config, probe=True)
|
||||
assert mode == "init-container"
|
||||
assert ready is False
|
||||
assert detail
|
||||
assert probed is True
|
||||
|
||||
|
||||
def test_status_runtime_mode_init_container_unreachable(monkeypatch, tmp_path) -> None:
|
||||
@ -638,10 +644,11 @@ def test_status_runtime_mode_init_container_unreachable(monkeypatch, tmp_path) -
|
||||
provisioner_url="http://provisioner:8002",
|
||||
)
|
||||
monkeypatch.setattr(lark_cli, "_probe_provisioner_capabilities", lambda _config: None)
|
||||
mode, ready, detail = lark_cli._resolve_sandbox_runtime_readiness(config, probe=True)
|
||||
mode, ready, detail, probed = lark_cli._resolve_sandbox_runtime_readiness(config, probe=True)
|
||||
assert mode == "init-container"
|
||||
assert ready is False
|
||||
assert detail
|
||||
assert probed is True
|
||||
|
||||
|
||||
def test_status_runtime_probe_skipped_when_not_requested(monkeypatch, tmp_path) -> None:
|
||||
@ -655,10 +662,32 @@ def test_status_runtime_probe_skipped_when_not_requested(monkeypatch, tmp_path)
|
||||
raise AssertionError("provisioner should not be probed when probe=False")
|
||||
|
||||
monkeypatch.setattr(lark_cli, "_probe_provisioner_capabilities", _fail)
|
||||
mode, ready, detail = lark_cli._resolve_sandbox_runtime_readiness(config, probe=False)
|
||||
mode, ready, detail, probed = lark_cli._resolve_sandbox_runtime_readiness(config, probe=False)
|
||||
assert mode == "init-container"
|
||||
assert ready is False
|
||||
assert detail is None
|
||||
assert probed is False
|
||||
|
||||
|
||||
def test_status_explicitly_reports_remote_runtime_probe_state(monkeypatch, tmp_path) -> None:
|
||||
_patch_paths(monkeypatch, tmp_path / "home")
|
||||
config = _config(tmp_path / "skills")
|
||||
config.sandbox = SimpleNamespace(
|
||||
use="deerflow.community.aio_sandbox:AioSandboxProvider",
|
||||
provisioner_url="http://provisioner:8002",
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
lark_cli,
|
||||
"_probe_provisioner_capabilities",
|
||||
lambda _config: {"lark_cli_init_image": True, "lark_cli_broker_image": False},
|
||||
)
|
||||
|
||||
unprobed = lark_cli.get_lark_integration_status("alice", config, check_runtime=False)
|
||||
probed = lark_cli.get_lark_integration_status("alice", config, check_runtime=True)
|
||||
|
||||
assert unprobed.sandbox_runtime_probed is False
|
||||
assert probed.sandbox_runtime_probed is True
|
||||
assert probed.sandbox_runtime_ready is True
|
||||
|
||||
|
||||
def _reset_broker_mode_cache() -> None:
|
||||
@ -938,6 +967,60 @@ def test_install_lark_integration_reports_content_change_on_reinstall(monkeypatc
|
||||
reset_skill_storage()
|
||||
|
||||
|
||||
def test_install_lark_integration_snapshots_credentials_under_lock(monkeypatch, tmp_path) -> None:
|
||||
reset_skill_storage()
|
||||
_patch_paths(monkeypatch, tmp_path / "home")
|
||||
skills_root = tmp_path / "skills"
|
||||
(skills_root / "public").mkdir(parents=True)
|
||||
(skills_root / "custom").mkdir()
|
||||
config = _config(skills_root)
|
||||
archive = _make_lark_cli_source_zip(tmp_path)
|
||||
|
||||
lock_state = {"active": False}
|
||||
snapshot_lock_states: list[bool] = []
|
||||
status_lock_states: list[bool] = []
|
||||
|
||||
class _CredentialLock:
|
||||
def __enter__(self):
|
||||
assert lock_state["active"] is False
|
||||
lock_state["active"] = True
|
||||
return self
|
||||
|
||||
def __exit__(self, *_args):
|
||||
lock_state["active"] = False
|
||||
|
||||
monkeypatch.setattr(lark_cli, "_lark_credential_lock", lambda _user_id: _CredentialLock())
|
||||
monkeypatch.setattr(lark_cli, "probe_lark_cli", lambda: lark_cli.LarkCliProbe(available=True, path="/usr/bin/lark-cli", version="v1.0.65"))
|
||||
|
||||
def _credential_snapshot(_user_id, *, verify_auth):
|
||||
snapshot_lock_states.append(lock_state["active"])
|
||||
return lark_cli._LarkCredentialSnapshot(
|
||||
app_config={"configured": True, "app_id": "cli_test", "brand": "feishu"},
|
||||
auth=lark_cli.LarkAuthProbe(status="authenticated", user="Alice", verified=False),
|
||||
)
|
||||
|
||||
monkeypatch.setattr(lark_cli, "_read_lark_credential_snapshot", _credential_snapshot)
|
||||
|
||||
real_status = lark_cli.get_lark_integration_status
|
||||
|
||||
def _status(_user_id, _config, **kwargs):
|
||||
status_lock_states.append(lock_state["active"])
|
||||
assert kwargs.get("check_runtime") is True
|
||||
assert kwargs.get("credential_snapshot") is not None
|
||||
return real_status(_user_id, _config, **kwargs)
|
||||
|
||||
monkeypatch.setattr(lark_cli, "get_lark_integration_status", _status)
|
||||
|
||||
result = lark_cli.install_lark_integration("alice", config, source_archive=archive)
|
||||
|
||||
assert result.success is True
|
||||
assert result.status.app_id == "cli_test"
|
||||
assert result.status.auth.status == "authenticated"
|
||||
assert snapshot_lock_states == [True]
|
||||
assert status_lock_states == [False]
|
||||
reset_skill_storage()
|
||||
|
||||
|
||||
def test_install_lark_integration_rejects_zip_slip_member(monkeypatch, tmp_path):
|
||||
_patch_paths(monkeypatch, tmp_path / "home")
|
||||
config = _config(tmp_path / "skills")
|
||||
@ -2212,6 +2295,299 @@ def test_complete_lark_auth_polls_device_code_and_returns_status(monkeypatch, tm
|
||||
}
|
||||
|
||||
|
||||
def test_complete_lark_auth_captures_status_under_lock_and_probes_runtime_after(monkeypatch, tmp_path) -> None:
|
||||
config = _config(tmp_path / "skills")
|
||||
lock_state = {"active": False}
|
||||
credential_state = {
|
||||
"app_id": "previous-app",
|
||||
"auth": lark_cli.LarkAuthProbe(status="not_authorized", user=None),
|
||||
}
|
||||
snapshot_lock_states: list[bool] = []
|
||||
status_lock_states: list[bool] = []
|
||||
|
||||
class _CredentialLock:
|
||||
def __enter__(self):
|
||||
assert lock_state["active"] is False
|
||||
lock_state["active"] = True
|
||||
|
||||
def __exit__(self, *_args):
|
||||
lock_state["active"] = False
|
||||
# Model another tab switching credentials as soon as this operation unlocks.
|
||||
credential_state["app_id"] = "next-app"
|
||||
credential_state["auth"] = lark_cli.LarkAuthProbe(status="not_authorized", user=None)
|
||||
|
||||
monkeypatch.setattr(lark_cli, "_lark_credential_lock", lambda _user_id: _CredentialLock())
|
||||
monkeypatch.setattr(lark_cli, "_require_lark_flow_generation_locked", lambda _user_id, generation: generation)
|
||||
monkeypatch.setattr(lark_cli, "_require_lark_cli_path", lambda: "/usr/bin/lark-cli")
|
||||
|
||||
def _complete_login(*_args, **_kwargs):
|
||||
assert lock_state["active"] is True
|
||||
credential_state["app_id"] = "completed-app"
|
||||
credential_state["auth"] = lark_cli.LarkAuthProbe(status="authenticated", user="Alice", verified=True)
|
||||
|
||||
monkeypatch.setattr(lark_cli, "_run_lark_cli_json", _complete_login)
|
||||
|
||||
def _credential_snapshot(_user_id, *, verify_auth):
|
||||
snapshot_lock_states.append(lock_state["active"])
|
||||
assert verify_auth is True
|
||||
return lark_cli._LarkCredentialSnapshot(
|
||||
app_config={"configured": True, "app_id": credential_state["app_id"], "brand": "feishu"},
|
||||
auth=credential_state["auth"],
|
||||
)
|
||||
|
||||
monkeypatch.setattr(lark_cli, "_read_lark_credential_snapshot", _credential_snapshot)
|
||||
|
||||
def _status(_user_id, _config, **kwargs):
|
||||
status_lock_states.append(lock_state["active"])
|
||||
assert kwargs.get("check_runtime") is True
|
||||
credential_snapshot = kwargs["credential_snapshot"]
|
||||
return lark_cli.LarkIntegrationStatus(
|
||||
installed=True,
|
||||
version="v1.0.65",
|
||||
manifest_version="v1.0.65",
|
||||
latest_available_version=None,
|
||||
runtime_version_mismatch=False,
|
||||
app_configured=True,
|
||||
app_id=credential_snapshot.app_config["app_id"],
|
||||
app_brand="feishu",
|
||||
skills_expected=27,
|
||||
skills_installed=27,
|
||||
installed_skills=("lark-doc",),
|
||||
enabled_skills=("lark-doc",),
|
||||
install_path="/tmp/lark",
|
||||
cli=lark_cli.LarkCliProbe(available=True),
|
||||
auth=credential_snapshot.auth,
|
||||
sandbox_runtime_mode="init-container",
|
||||
sandbox_runtime_probed=True,
|
||||
sandbox_runtime_ready=True,
|
||||
)
|
||||
|
||||
monkeypatch.setattr(lark_cli, "get_lark_integration_status", _status)
|
||||
|
||||
result = lark_cli.complete_lark_auth(
|
||||
"alice",
|
||||
config,
|
||||
device_code="device-code",
|
||||
generation="auth-generation",
|
||||
)
|
||||
|
||||
assert result.success is True
|
||||
assert result.status.app_id == "completed-app"
|
||||
assert result.status.auth.status == "authenticated"
|
||||
assert result.status.sandbox_runtime_ready is True
|
||||
assert snapshot_lock_states == [True]
|
||||
assert status_lock_states == [False]
|
||||
|
||||
|
||||
def test_complete_lark_auth_status_ignores_credential_changes_during_runtime_probe(monkeypatch, tmp_path) -> None:
|
||||
"""End-to-end with the real status builder: a credential change landing while
|
||||
the unlocked runtime probe runs must not turn the completed login into
|
||||
success=false or mix newer credential state into the response."""
|
||||
_patch_paths(monkeypatch, tmp_path / "home")
|
||||
skills_root = tmp_path / "skills"
|
||||
(skills_root / "public").mkdir(parents=True)
|
||||
(skills_root / "custom").mkdir()
|
||||
config = _config(skills_root)
|
||||
|
||||
credential_state = {
|
||||
"app": {"configured": True, "app_id": "completed-app", "brand": "feishu"},
|
||||
"auth": lark_cli.LarkAuthProbe(status="authenticated", user="Alice", verified=True),
|
||||
}
|
||||
monkeypatch.setattr(lark_cli, "read_lark_app_config", lambda _user_id: dict(credential_state["app"]))
|
||||
monkeypatch.setattr(lark_cli, "probe_lark_auth", lambda _user_id, *, verify=False: credential_state["auth"])
|
||||
monkeypatch.setattr(lark_cli, "probe_lark_cli", lambda: lark_cli.LarkCliProbe(available=True, version="1.0.65"))
|
||||
monkeypatch.setattr(lark_cli, "_require_lark_cli_path", lambda: "/usr/bin/lark-cli")
|
||||
monkeypatch.setattr(lark_cli, "_run_lark_cli_json", lambda *_args, **_kwargs: {})
|
||||
|
||||
runtime_probes: list[bool] = []
|
||||
|
||||
def _runtime_probe(_config, *, probe):
|
||||
# A concurrent app switch clears these tokens while the runtime probe runs.
|
||||
credential_state["app"] = {"configured": True, "app_id": "racer-app", "brand": "feishu"}
|
||||
credential_state["auth"] = lark_cli.LarkAuthProbe(status="not_authorized", message="tokens cleared")
|
||||
runtime_probes.append(probe)
|
||||
return "init-container", True, None, True
|
||||
|
||||
monkeypatch.setattr(lark_cli, "_resolve_sandbox_runtime_readiness", _runtime_probe)
|
||||
|
||||
generation = _advance_lark_flow()
|
||||
result = lark_cli.complete_lark_auth("alice", config, device_code="device-code", generation=generation)
|
||||
|
||||
assert runtime_probes == [True]
|
||||
assert result.success is True
|
||||
assert result.status.app_id == "completed-app"
|
||||
assert result.status.auth.status == "authenticated"
|
||||
assert result.status.sandbox_runtime_probed is True
|
||||
assert result.status.sandbox_runtime_ready is True
|
||||
|
||||
|
||||
def test_complete_lark_config_captures_status_under_lock_and_probes_runtime_after(monkeypatch, tmp_path) -> None:
|
||||
config = _config(tmp_path / "skills")
|
||||
lock_state = {"active": False}
|
||||
credential_state = {
|
||||
"app_id": "previous-app",
|
||||
"auth": lark_cli.LarkAuthProbe(status="not_authorized", user=None),
|
||||
}
|
||||
snapshot_lock_states: list[bool] = []
|
||||
status_lock_states: list[bool] = []
|
||||
|
||||
class _CredentialLock:
|
||||
def __enter__(self):
|
||||
assert lock_state["active"] is False
|
||||
lock_state["active"] = True
|
||||
|
||||
def __exit__(self, *_args):
|
||||
lock_state["active"] = False
|
||||
# Model another tab switching credentials as soon as this operation unlocks.
|
||||
credential_state["app_id"] = "next-app"
|
||||
credential_state["auth"] = lark_cli.LarkAuthProbe(status="not_authorized", user=None)
|
||||
|
||||
monkeypatch.setattr(lark_cli, "_lark_credential_lock", lambda _user_id: _CredentialLock())
|
||||
monkeypatch.setattr(lark_cli, "_require_lark_flow_generation_locked", lambda _user_id, generation: generation)
|
||||
monkeypatch.setattr(
|
||||
lark_cli,
|
||||
"_poll_lark_app_registration",
|
||||
lambda **_kwargs: {"client_id": "cli_registered", "client_secret": "registered-secret"},
|
||||
)
|
||||
|
||||
def _replace(_user_id, *, app_id, app_secret, brand):
|
||||
assert lock_state["active"] is True
|
||||
credential_state["app_id"] = app_id
|
||||
credential_state["auth"] = lark_cli.LarkAuthProbe(status="not_authorized", message="authorize the new app")
|
||||
|
||||
monkeypatch.setattr(lark_cli, "_replace_lark_app_credentials_locked", _replace)
|
||||
|
||||
def _credential_snapshot(_user_id, *, verify_auth):
|
||||
snapshot_lock_states.append(lock_state["active"])
|
||||
assert verify_auth is False
|
||||
return lark_cli._LarkCredentialSnapshot(
|
||||
app_config={"configured": True, "app_id": credential_state["app_id"], "brand": "feishu"},
|
||||
auth=credential_state["auth"],
|
||||
)
|
||||
|
||||
monkeypatch.setattr(lark_cli, "_read_lark_credential_snapshot", _credential_snapshot)
|
||||
|
||||
def _status(_user_id, _config, **kwargs):
|
||||
status_lock_states.append(lock_state["active"])
|
||||
assert kwargs.get("check_runtime") is True
|
||||
credential_snapshot = kwargs["credential_snapshot"]
|
||||
return lark_cli.LarkIntegrationStatus(
|
||||
installed=True,
|
||||
version="v1.0.65",
|
||||
manifest_version="v1.0.65",
|
||||
latest_available_version=None,
|
||||
runtime_version_mismatch=False,
|
||||
app_configured=True,
|
||||
app_id=credential_snapshot.app_config["app_id"],
|
||||
app_brand="feishu",
|
||||
skills_expected=27,
|
||||
skills_installed=27,
|
||||
installed_skills=("lark-doc",),
|
||||
enabled_skills=("lark-doc",),
|
||||
install_path="/tmp/lark",
|
||||
cli=lark_cli.LarkCliProbe(available=True),
|
||||
auth=credential_snapshot.auth,
|
||||
sandbox_runtime_mode="init-container",
|
||||
sandbox_runtime_probed=True,
|
||||
sandbox_runtime_ready=True,
|
||||
)
|
||||
|
||||
monkeypatch.setattr(lark_cli, "get_lark_integration_status", _status)
|
||||
|
||||
result = lark_cli.complete_lark_config(
|
||||
"alice",
|
||||
config,
|
||||
device_code="device-code",
|
||||
generation="config-generation",
|
||||
)
|
||||
|
||||
assert result.success is True
|
||||
assert result.status.app_id == "cli_registered"
|
||||
assert result.status.sandbox_runtime_ready is True
|
||||
assert snapshot_lock_states == [True]
|
||||
assert status_lock_states == [False]
|
||||
|
||||
|
||||
def test_set_lark_app_credentials_captures_status_under_lock_and_probes_runtime_after(monkeypatch, tmp_path) -> None:
|
||||
config = _config(tmp_path / "skills")
|
||||
lock_state = {"active": False}
|
||||
credential_state = {
|
||||
"app_id": "previous-app",
|
||||
"auth": lark_cli.LarkAuthProbe(status="authenticated", user="Alice"),
|
||||
}
|
||||
snapshot_lock_states: list[bool] = []
|
||||
status_lock_states: list[bool] = []
|
||||
|
||||
class _CredentialLock:
|
||||
def __enter__(self):
|
||||
assert lock_state["active"] is False
|
||||
lock_state["active"] = True
|
||||
|
||||
def __exit__(self, *_args):
|
||||
lock_state["active"] = False
|
||||
# Model another tab switching credentials as soon as this operation unlocks.
|
||||
credential_state["app_id"] = "racer-app"
|
||||
credential_state["auth"] = lark_cli.LarkAuthProbe(status="authenticated", user="Racer")
|
||||
|
||||
monkeypatch.setattr(lark_cli, "_lark_credential_lock", lambda _user_id: _CredentialLock())
|
||||
monkeypatch.setattr(lark_cli, "_validate_lark_app_credentials_with_cli", lambda **kwargs: None)
|
||||
monkeypatch.setattr(lark_cli, "_advance_lark_flow_generation_locked", lambda _user_id: "switch-generation")
|
||||
|
||||
def _replace(_user_id, *, app_id, app_secret, brand):
|
||||
assert lock_state["active"] is True
|
||||
credential_state["app_id"] = app_id
|
||||
credential_state["auth"] = lark_cli.LarkAuthProbe(status="not_authorized", message="prior token revoked")
|
||||
|
||||
monkeypatch.setattr(lark_cli, "_replace_lark_app_credentials_locked", _replace)
|
||||
|
||||
def _credential_snapshot(_user_id, *, verify_auth):
|
||||
snapshot_lock_states.append(lock_state["active"])
|
||||
assert verify_auth is False
|
||||
return lark_cli._LarkCredentialSnapshot(
|
||||
app_config={"configured": True, "app_id": credential_state["app_id"], "brand": "feishu"},
|
||||
auth=credential_state["auth"],
|
||||
)
|
||||
|
||||
monkeypatch.setattr(lark_cli, "_read_lark_credential_snapshot", _credential_snapshot)
|
||||
|
||||
def _status(_user_id, _config, **kwargs):
|
||||
status_lock_states.append(lock_state["active"])
|
||||
assert kwargs.get("check_runtime") is True
|
||||
credential_snapshot = kwargs["credential_snapshot"]
|
||||
return lark_cli.LarkIntegrationStatus(
|
||||
installed=True,
|
||||
version="v1.0.65",
|
||||
manifest_version="v1.0.65",
|
||||
latest_available_version=None,
|
||||
runtime_version_mismatch=False,
|
||||
app_configured=True,
|
||||
app_id=credential_snapshot.app_config["app_id"],
|
||||
app_brand="feishu",
|
||||
skills_expected=27,
|
||||
skills_installed=27,
|
||||
installed_skills=("lark-doc",),
|
||||
enabled_skills=("lark-doc",),
|
||||
install_path="/tmp/lark",
|
||||
cli=lark_cli.LarkCliProbe(available=True),
|
||||
auth=credential_snapshot.auth,
|
||||
sandbox_runtime_mode="init-container",
|
||||
sandbox_runtime_probed=True,
|
||||
sandbox_runtime_ready=True,
|
||||
)
|
||||
|
||||
monkeypatch.setattr(lark_cli, "get_lark_integration_status", _status)
|
||||
|
||||
result = lark_cli.set_lark_app_credentials("alice", config, app_id="cli_new", app_secret="new-secret")
|
||||
|
||||
assert result.success is True
|
||||
assert result.generation == "switch-generation"
|
||||
assert result.status.app_id == "cli_new"
|
||||
assert result.status.auth.status == "not_authorized"
|
||||
assert result.status.sandbox_runtime_ready is True
|
||||
assert snapshot_lock_states == [True]
|
||||
assert status_lock_states == [False]
|
||||
|
||||
|
||||
def test_complete_lark_auth_accepts_short_automatic_poll_timeout(monkeypatch, tmp_path) -> None:
|
||||
assert "wait_timeout_seconds" in inspect.signature(lark_cli.complete_lark_auth).parameters
|
||||
_patch_paths(monkeypatch, tmp_path / "home")
|
||||
@ -2576,6 +2952,7 @@ def test_lark_status_is_available_to_authenticated_users(monkeypatch, tmp_path):
|
||||
install_path="/tmp/lark-cli",
|
||||
cli=lark_cli.LarkCliProbe(available=False, error="missing"),
|
||||
auth=lark_cli.LarkAuthProbe(status="unavailable", message="missing"),
|
||||
sandbox_runtime_probed=True,
|
||||
),
|
||||
)
|
||||
|
||||
@ -2584,6 +2961,7 @@ def test_lark_status_is_available_to_authenticated_users(monkeypatch, tmp_path):
|
||||
|
||||
assert response.status_code == 200
|
||||
assert response.json()["installed"] is False
|
||||
assert response.json()["sandbox_runtime_probed"] is True
|
||||
|
||||
|
||||
def _status_with_host_paths() -> lark_cli.LarkIntegrationStatus:
|
||||
|
||||
@ -29,6 +29,7 @@ export function POST() {
|
||||
verified: true,
|
||||
},
|
||||
sandbox_runtime_mode: "init-container",
|
||||
sandbox_runtime_probed: true,
|
||||
sandbox_runtime_ready: true,
|
||||
sandbox_runtime_detail: null,
|
||||
},
|
||||
|
||||
@ -29,6 +29,7 @@ export function POST() {
|
||||
verified: false,
|
||||
},
|
||||
sandbox_runtime_mode: "none",
|
||||
sandbox_runtime_probed: true,
|
||||
sandbox_runtime_ready: false,
|
||||
sandbox_runtime_detail: null,
|
||||
},
|
||||
|
||||
@ -29,6 +29,7 @@ export function POST() {
|
||||
verified: false,
|
||||
},
|
||||
sandbox_runtime_mode: "none",
|
||||
sandbox_runtime_probed: true,
|
||||
sandbox_runtime_ready: false,
|
||||
sandbox_runtime_detail: null,
|
||||
},
|
||||
|
||||
@ -32,6 +32,7 @@ export function POST() {
|
||||
verified: false,
|
||||
},
|
||||
sandbox_runtime_mode: "none",
|
||||
sandbox_runtime_probed: true,
|
||||
sandbox_runtime_ready: false,
|
||||
sandbox_runtime_detail: null,
|
||||
},
|
||||
|
||||
@ -25,6 +25,7 @@ const status = {
|
||||
verified: false,
|
||||
},
|
||||
sandbox_runtime_mode: "none",
|
||||
sandbox_runtime_probed: true,
|
||||
sandbox_runtime_ready: false,
|
||||
sandbox_runtime_detail: null,
|
||||
};
|
||||
|
||||
@ -28,6 +28,7 @@ import { useAuth } from "@/core/auth/AuthProvider";
|
||||
import { writeTextToClipboard } from "@/core/clipboard";
|
||||
import { useI18n } from "@/core/i18n/hooks";
|
||||
import {
|
||||
cacheLarkMutationStatus,
|
||||
larkIntegrationQueryKey,
|
||||
LarkIntegrationRequestError,
|
||||
type LarkAuthStartRequest,
|
||||
@ -303,7 +304,7 @@ function LarkIntegrationCard() {
|
||||
{
|
||||
onSuccess: (result) => {
|
||||
if (!isActiveFlow(generation)) return;
|
||||
queryClient.setQueryData(larkIntegrationQueryKey, result.status);
|
||||
void cacheLarkMutationStatus(queryClient, result.status);
|
||||
toast.success(t.settings.integrations.lark.connectionReady);
|
||||
setPendingFlow(null);
|
||||
startUserAuth(
|
||||
@ -381,7 +382,7 @@ function LarkIntegrationCard() {
|
||||
{
|
||||
onSuccess: (result) => {
|
||||
if (!isActiveFlow(generation)) return;
|
||||
queryClient.setQueryData(larkIntegrationQueryKey, result.status);
|
||||
void cacheLarkMutationStatus(queryClient, result.status);
|
||||
toast.success(t.settings.integrations.lark.changeAppSwitched);
|
||||
setChangeAppSecret("");
|
||||
setShowChangeApp(false);
|
||||
@ -486,7 +487,7 @@ function LarkIntegrationCard() {
|
||||
if (automatic && attemptId !== authAttemptIdRef.current) {
|
||||
return;
|
||||
}
|
||||
queryClient.setQueryData(larkIntegrationQueryKey, result.status);
|
||||
void cacheLarkMutationStatus(queryClient, result.status);
|
||||
if (result.success) {
|
||||
clearAuthRetryTimer();
|
||||
toast.success(result.message, toastOptions);
|
||||
|
||||
@ -36,6 +36,18 @@ async function readErrorDetail(response: Response): Promise<string> {
|
||||
return data.detail ?? `HTTP ${response.status}: ${response.statusText}`;
|
||||
}
|
||||
|
||||
function normalizeLarkIntegrationStatus(
|
||||
status: LarkIntegrationStatus,
|
||||
): LarkIntegrationStatus {
|
||||
// Backends predating the flag omit it entirely; an absent flag means the
|
||||
// runtime readiness was not reported as evaluated, so default to false and
|
||||
// let the mutation cache preserve the last authoritative runtime fields.
|
||||
return {
|
||||
...status,
|
||||
sandbox_runtime_probed: status.sandbox_runtime_probed ?? false,
|
||||
};
|
||||
}
|
||||
|
||||
export async function loadLarkIntegrationStatus(
|
||||
signal?: AbortSignal,
|
||||
): Promise<LarkIntegrationStatus> {
|
||||
@ -49,7 +61,7 @@ export async function loadLarkIntegrationStatus(
|
||||
await readErrorDetail(response),
|
||||
);
|
||||
}
|
||||
return response.json();
|
||||
return normalizeLarkIntegrationStatus(await response.json());
|
||||
}
|
||||
|
||||
export async function installLarkIntegration(): Promise<LarkInstallResponse> {
|
||||
@ -65,7 +77,8 @@ export async function installLarkIntegration(): Promise<LarkInstallResponse> {
|
||||
await readErrorDetail(response),
|
||||
);
|
||||
}
|
||||
return response.json();
|
||||
const data = (await response.json()) as LarkInstallResponse;
|
||||
return { ...data, status: normalizeLarkIntegrationStatus(data.status) };
|
||||
}
|
||||
|
||||
export async function startLarkAuthorization(
|
||||
@ -131,7 +144,8 @@ export async function completeLarkConfiguration(
|
||||
await readErrorDetail(response),
|
||||
);
|
||||
}
|
||||
return response.json();
|
||||
const data = (await response.json()) as LarkConfigCompleteResponse;
|
||||
return { ...data, status: normalizeLarkIntegrationStatus(data.status) };
|
||||
}
|
||||
|
||||
export async function setLarkAppCredentials(
|
||||
@ -153,7 +167,8 @@ export async function setLarkAppCredentials(
|
||||
await readErrorDetail(response),
|
||||
);
|
||||
}
|
||||
return response.json();
|
||||
const data = (await response.json()) as LarkConfigCompleteResponse;
|
||||
return { ...data, status: normalizeLarkIntegrationStatus(data.status) };
|
||||
}
|
||||
|
||||
export async function completeLarkAuthorization(
|
||||
@ -175,5 +190,6 @@ export async function completeLarkAuthorization(
|
||||
await readErrorDetail(response),
|
||||
);
|
||||
}
|
||||
return response.json();
|
||||
const data = (await response.json()) as LarkAuthCompleteResponse;
|
||||
return { ...data, status: normalizeLarkIntegrationStatus(data.status) };
|
||||
}
|
||||
|
||||
45
frontend/src/core/integrations/lark/cache.ts
Normal file
45
frontend/src/core/integrations/lark/cache.ts
Normal file
@ -0,0 +1,45 @@
|
||||
import { type QueryClient } from "@tanstack/react-query";
|
||||
|
||||
import { type LarkIntegrationStatus } from "./types";
|
||||
|
||||
export const larkIntegrationQueryKey = ["integrations", "lark"] as const;
|
||||
|
||||
type LarkIntegrationQueryClient = Pick<
|
||||
QueryClient,
|
||||
"cancelQueries" | "setQueryData" | "invalidateQueries"
|
||||
>;
|
||||
|
||||
function hasUnprobedRuntime(status: LarkIntegrationStatus): boolean {
|
||||
return !status.sandbox_runtime_probed;
|
||||
}
|
||||
|
||||
export async function cacheLarkMutationStatus(
|
||||
queryClient: LarkIntegrationQueryClient,
|
||||
status: LarkIntegrationStatus,
|
||||
): Promise<void> {
|
||||
// Await the cancel so an in-flight status read fully settles before the
|
||||
// mutation snapshot lands: a fire-and-forget cancel loses to a GET whose
|
||||
// fetch already resolved but whose cache write is still queued, and that
|
||||
// stale write would clobber the fresher mutation status.
|
||||
await queryClient.cancelQueries({ queryKey: larkIntegrationQueryKey });
|
||||
queryClient.setQueryData<LarkIntegrationStatus>(
|
||||
larkIntegrationQueryKey,
|
||||
(current) =>
|
||||
current && hasUnprobedRuntime(status)
|
||||
? {
|
||||
...status,
|
||||
sandbox_runtime_mode: current.sandbox_runtime_mode,
|
||||
sandbox_runtime_probed: current.sandbox_runtime_probed,
|
||||
sandbox_runtime_ready: current.sandbox_runtime_ready,
|
||||
sandbox_runtime_detail: current.sandbox_runtime_detail,
|
||||
}
|
||||
: status,
|
||||
);
|
||||
if (hasUnprobedRuntime(status)) {
|
||||
// Unprobed means the response came from a backend that predates the
|
||||
// field. The cancelled in-flight GET carried the authoritative runtime
|
||||
// fields, so refetch instead of risking the conservative fallback
|
||||
// lingering until the next focus/remount.
|
||||
void queryClient.invalidateQueries({ queryKey: larkIntegrationQueryKey });
|
||||
}
|
||||
}
|
||||
@ -9,8 +9,7 @@ import {
|
||||
startLarkAuthorization,
|
||||
startLarkConfiguration,
|
||||
} from "./api";
|
||||
|
||||
export const larkIntegrationQueryKey = ["integrations", "lark"] as const;
|
||||
import { cacheLarkMutationStatus, larkIntegrationQueryKey } from "./cache";
|
||||
|
||||
export function useLarkIntegrationStatus() {
|
||||
return useQuery({
|
||||
@ -24,7 +23,7 @@ export function useInstallLarkIntegration() {
|
||||
return useMutation({
|
||||
mutationFn: installLarkIntegration,
|
||||
onSuccess: async (result) => {
|
||||
queryClient.setQueryData(larkIntegrationQueryKey, result.status);
|
||||
await cacheLarkMutationStatus(queryClient, result.status);
|
||||
await queryClient.invalidateQueries({
|
||||
queryKey: larkIntegrationQueryKey,
|
||||
});
|
||||
|
||||
@ -1,3 +1,4 @@
|
||||
export * from "./api";
|
||||
export * from "./cache";
|
||||
export * from "./hooks";
|
||||
export * from "./types";
|
||||
|
||||
@ -40,6 +40,7 @@ export interface LarkIntegrationStatus {
|
||||
cli: LarkCliProbe;
|
||||
auth: LarkAuthProbe;
|
||||
sandbox_runtime_mode: LarkSandboxRuntimeMode;
|
||||
sandbox_runtime_probed: boolean;
|
||||
sandbox_runtime_ready: boolean;
|
||||
sandbox_runtime_detail: string | null;
|
||||
}
|
||||
|
||||
@ -30,6 +30,7 @@ function configuredLarkStatus() {
|
||||
verified: true,
|
||||
},
|
||||
sandbox_runtime_mode: "none",
|
||||
sandbox_runtime_probed: true,
|
||||
sandbox_runtime_ready: false,
|
||||
sandbox_runtime_detail: null,
|
||||
};
|
||||
|
||||
@ -381,7 +381,9 @@ export function mockLangGraphAPI(page: Page, options?: MockAPIOptions) {
|
||||
sandbox_runtime_mode: "init-container" as
|
||||
| "none"
|
||||
| "gateway-download"
|
||||
| "init-container",
|
||||
| "init-container"
|
||||
| "broker",
|
||||
sandbox_runtime_probed: true,
|
||||
sandbox_runtime_ready: false,
|
||||
sandbox_runtime_detail:
|
||||
"The provisioner has no lark-cli init image configured (LARK_CLI_INIT_IMAGE)." as
|
||||
@ -1969,6 +1971,7 @@ export function mockLangGraphAPI(page: Page, options?: MockAPIOptions) {
|
||||
verified: false,
|
||||
},
|
||||
sandbox_runtime_mode: "init-container",
|
||||
sandbox_runtime_probed: true,
|
||||
sandbox_runtime_ready: true,
|
||||
sandbox_runtime_detail: null,
|
||||
};
|
||||
|
||||
@ -55,6 +55,7 @@ describe("lark integration api", () => {
|
||||
cli: { available: false, path: null, version: null, error: "missing" },
|
||||
auth: { status: "unavailable", message: "missing", user: null },
|
||||
sandbox_runtime_mode: "init-container",
|
||||
sandbox_runtime_probed: true,
|
||||
sandbox_runtime_ready: false,
|
||||
sandbox_runtime_detail: "init image not configured",
|
||||
}),
|
||||
@ -66,6 +67,7 @@ describe("lark integration api", () => {
|
||||
installed: false,
|
||||
version: "v1.0.65",
|
||||
sandbox_runtime_mode: "init-container",
|
||||
sandbox_runtime_probed: true,
|
||||
sandbox_runtime_ready: false,
|
||||
});
|
||||
expect(mockedFetch).toHaveBeenCalledWith(
|
||||
@ -384,4 +386,29 @@ describe("lark integration api", () => {
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
test("status load defaults sandbox_runtime_probed to false when the backend omits it", async () => {
|
||||
mockedFetch.mockResolvedValueOnce(
|
||||
jsonResponse(200, { installed: false, version: "v1.0.65" }),
|
||||
);
|
||||
|
||||
const result = await loadLarkIntegrationStatus();
|
||||
|
||||
expect(result.sandbox_runtime_probed).toBe(false);
|
||||
});
|
||||
|
||||
test("mutation responses default sandbox_runtime_probed to false when the backend omits it", async () => {
|
||||
mockedFetch.mockResolvedValueOnce(
|
||||
jsonResponse(200, {
|
||||
success: true,
|
||||
installed_skills: ["lark-doc"],
|
||||
message: "Installed 1 Lark/Feishu skills.",
|
||||
status: { installed: true, version: "v1.0.65" },
|
||||
}),
|
||||
);
|
||||
|
||||
const result = await installLarkIntegration();
|
||||
|
||||
expect(result.status.sandbox_runtime_probed).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
156
frontend/tests/unit/core/integrations/lark/cache.test.ts
Normal file
156
frontend/tests/unit/core/integrations/lark/cache.test.ts
Normal file
@ -0,0 +1,156 @@
|
||||
import { expect, rs, test } from "@rstest/core";
|
||||
import { QueryClient } from "@tanstack/react-query";
|
||||
|
||||
import { cacheLarkMutationStatus } from "@/core/integrations/lark/cache";
|
||||
import { type LarkIntegrationStatus } from "@/core/integrations/lark/types";
|
||||
|
||||
const status: LarkIntegrationStatus = {
|
||||
installed: true,
|
||||
version: "v1.0.65",
|
||||
manifest_version: "v1.0.65",
|
||||
latest_available_version: null,
|
||||
runtime_version_mismatch: false,
|
||||
app_configured: true,
|
||||
app_id: "cli_mock",
|
||||
app_brand: "feishu",
|
||||
skills_expected: 27,
|
||||
skills_installed: 27,
|
||||
installed_skills: ["lark-doc"],
|
||||
enabled_skills: ["lark-doc"],
|
||||
install_path: "/tmp/lark",
|
||||
cli: {
|
||||
available: true,
|
||||
path: "/usr/bin/lark-cli",
|
||||
version: "v1.0.65",
|
||||
error: null,
|
||||
},
|
||||
auth: {
|
||||
status: "authenticated",
|
||||
message: null,
|
||||
user: "Alice",
|
||||
verified: true,
|
||||
},
|
||||
sandbox_runtime_mode: "init-container",
|
||||
sandbox_runtime_probed: true,
|
||||
sandbox_runtime_ready: true,
|
||||
sandbox_runtime_detail: null,
|
||||
};
|
||||
|
||||
test("mutation status cancels stale reads before populating an empty cache", async () => {
|
||||
const queryClient = new QueryClient();
|
||||
const cancelQueries = rs.spyOn(queryClient, "cancelQueries");
|
||||
|
||||
await cacheLarkMutationStatus(queryClient, status);
|
||||
|
||||
expect(cancelQueries).toHaveBeenCalledWith({
|
||||
queryKey: ["integrations", "lark"],
|
||||
});
|
||||
expect(queryClient.getQueryData(["integrations", "lark"])).toEqual(status);
|
||||
});
|
||||
|
||||
test("unprobed mutation status preserves runtime fields from the authoritative GET", async () => {
|
||||
const queryClient = new QueryClient();
|
||||
const invalidateQueries = rs.spyOn(queryClient, "invalidateQueries");
|
||||
queryClient.setQueryData(["integrations", "lark"], {
|
||||
...status,
|
||||
auth: { ...status.auth, verified: false },
|
||||
});
|
||||
const mutationStatus: LarkIntegrationStatus = {
|
||||
...status,
|
||||
// Different mode from the cached status, so the assertions below prove
|
||||
// the runtime fields came from the cache rather than the mutation.
|
||||
sandbox_runtime_mode: "gateway-download",
|
||||
sandbox_runtime_probed: false,
|
||||
sandbox_runtime_ready: false,
|
||||
sandbox_runtime_detail: null,
|
||||
};
|
||||
|
||||
await cacheLarkMutationStatus(queryClient, mutationStatus);
|
||||
|
||||
expect(
|
||||
queryClient.getQueryData<LarkIntegrationStatus>(["integrations", "lark"]),
|
||||
).toMatchObject({
|
||||
auth: { verified: true },
|
||||
sandbox_runtime_mode: "init-container",
|
||||
sandbox_runtime_probed: true,
|
||||
sandbox_runtime_ready: true,
|
||||
sandbox_runtime_detail: null,
|
||||
});
|
||||
// Unprobed mutation responses self-heal: the cancelled in-flight GET is
|
||||
// refetched so the conservative fallback cannot linger.
|
||||
expect(invalidateQueries).toHaveBeenCalledWith({
|
||||
queryKey: ["integrations", "lark"],
|
||||
});
|
||||
});
|
||||
|
||||
test("missing sandbox_runtime_probed (older backend) is treated as unprobed", async () => {
|
||||
const queryClient = new QueryClient();
|
||||
const invalidateQueries = rs.spyOn(queryClient, "invalidateQueries");
|
||||
queryClient.setQueryData(["integrations", "lark"], status);
|
||||
// Simulate a pre-field backend response: the key is absent at runtime even
|
||||
// though the type marks it required.
|
||||
const mutationStatus = { ...status } as Partial<LarkIntegrationStatus>;
|
||||
delete mutationStatus.sandbox_runtime_probed;
|
||||
|
||||
await cacheLarkMutationStatus(
|
||||
queryClient,
|
||||
mutationStatus as LarkIntegrationStatus,
|
||||
);
|
||||
|
||||
expect(
|
||||
queryClient.getQueryData<LarkIntegrationStatus>(["integrations", "lark"]),
|
||||
).toMatchObject({
|
||||
sandbox_runtime_mode: "init-container",
|
||||
sandbox_runtime_probed: true,
|
||||
sandbox_runtime_ready: true,
|
||||
});
|
||||
expect(invalidateQueries).toHaveBeenCalledWith({
|
||||
queryKey: ["integrations", "lark"],
|
||||
});
|
||||
});
|
||||
|
||||
test("explicitly probed unready status does not depend on detail text", async () => {
|
||||
const queryClient = new QueryClient();
|
||||
queryClient.setQueryData(["integrations", "lark"], status);
|
||||
const mutationStatus: LarkIntegrationStatus = {
|
||||
...status,
|
||||
sandbox_runtime_probed: true,
|
||||
sandbox_runtime_ready: false,
|
||||
sandbox_runtime_detail: null,
|
||||
};
|
||||
|
||||
await cacheLarkMutationStatus(queryClient, mutationStatus);
|
||||
|
||||
expect(
|
||||
queryClient.getQueryData<LarkIntegrationStatus>(["integrations", "lark"]),
|
||||
).toMatchObject({
|
||||
sandbox_runtime_probed: true,
|
||||
sandbox_runtime_ready: false,
|
||||
sandbox_runtime_detail: null,
|
||||
});
|
||||
});
|
||||
|
||||
test("probed mutation status replaces older runtime fields", async () => {
|
||||
const queryClient = new QueryClient();
|
||||
queryClient.setQueryData(["integrations", "lark"], {
|
||||
...status,
|
||||
sandbox_runtime_ready: false,
|
||||
sandbox_runtime_detail: "provisioner was unavailable",
|
||||
});
|
||||
const mutationStatus: LarkIntegrationStatus = {
|
||||
...status,
|
||||
sandbox_runtime_mode: "broker",
|
||||
sandbox_runtime_probed: true,
|
||||
};
|
||||
|
||||
await cacheLarkMutationStatus(queryClient, mutationStatus);
|
||||
|
||||
expect(
|
||||
queryClient.getQueryData<LarkIntegrationStatus>(["integrations", "lark"]),
|
||||
).toMatchObject({
|
||||
sandbox_runtime_mode: "broker",
|
||||
sandbox_runtime_probed: true,
|
||||
sandbox_runtime_ready: true,
|
||||
sandbox_runtime_detail: null,
|
||||
});
|
||||
});
|
||||
@ -0,0 +1,91 @@
|
||||
import { afterEach, describe, expect, it, rs } from "@rstest/core";
|
||||
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
|
||||
import { act, cleanup, renderHook } from "@testing-library/react";
|
||||
import type { PropsWithChildren } from "react";
|
||||
|
||||
rs.mock("@/core/integrations/lark/api", () => ({
|
||||
completeLarkAuthorization: rs.fn(),
|
||||
completeLarkConfiguration: rs.fn(),
|
||||
installLarkIntegration: rs.fn(),
|
||||
loadLarkIntegrationStatus: rs.fn(),
|
||||
setLarkAppCredentials: rs.fn(),
|
||||
startLarkAuthorization: rs.fn(),
|
||||
startLarkConfiguration: rs.fn(),
|
||||
}));
|
||||
|
||||
import { installLarkIntegration } from "@/core/integrations/lark/api";
|
||||
import { useInstallLarkIntegration } from "@/core/integrations/lark/hooks";
|
||||
import type { LarkIntegrationStatus } from "@/core/integrations/lark/types";
|
||||
|
||||
const mockedInstall = rs.mocked(installLarkIntegration);
|
||||
|
||||
const status: LarkIntegrationStatus = {
|
||||
installed: true,
|
||||
version: "v1.0.65",
|
||||
manifest_version: "v1.0.65",
|
||||
latest_available_version: null,
|
||||
runtime_version_mismatch: false,
|
||||
app_configured: true,
|
||||
app_id: "cli_mock",
|
||||
app_brand: "feishu",
|
||||
skills_expected: 27,
|
||||
skills_installed: 27,
|
||||
installed_skills: ["lark-doc"],
|
||||
enabled_skills: ["lark-doc"],
|
||||
install_path: "/tmp/lark",
|
||||
cli: {
|
||||
available: true,
|
||||
path: "/usr/bin/lark-cli",
|
||||
version: "v1.0.65",
|
||||
error: null,
|
||||
},
|
||||
auth: {
|
||||
status: "authenticated",
|
||||
message: null,
|
||||
user: "Alice",
|
||||
verified: true,
|
||||
},
|
||||
sandbox_runtime_mode: "init-container",
|
||||
sandbox_runtime_probed: true,
|
||||
sandbox_runtime_ready: true,
|
||||
sandbox_runtime_detail: null,
|
||||
};
|
||||
|
||||
afterEach(() => {
|
||||
cleanup();
|
||||
mockedInstall.mockReset();
|
||||
});
|
||||
|
||||
describe("useInstallLarkIntegration", () => {
|
||||
it("writes the mutation status through cacheLarkMutationStatus", async () => {
|
||||
// Regression guard: the install mutation must cancel stale in-flight
|
||||
// reads (i.e. go through cacheLarkMutationStatus) instead of a bare
|
||||
// setQueryData, so version-skewed unprobed responses cannot clobber the
|
||||
// authoritative runtime fields.
|
||||
mockedInstall.mockResolvedValueOnce({
|
||||
success: true,
|
||||
installed_skills: ["lark-doc"],
|
||||
message: "Installed 1 Lark/Feishu skills.",
|
||||
status,
|
||||
});
|
||||
const queryClient = new QueryClient({
|
||||
defaultOptions: { queries: { retry: false } },
|
||||
});
|
||||
const cancelQueries = rs.spyOn(queryClient, "cancelQueries");
|
||||
const wrapper = ({ children }: PropsWithChildren) => (
|
||||
<QueryClientProvider client={queryClient}>{children}</QueryClientProvider>
|
||||
);
|
||||
const { result } = renderHook(() => useInstallLarkIntegration(), {
|
||||
wrapper,
|
||||
});
|
||||
|
||||
await act(async () => {
|
||||
await result.current.mutateAsync();
|
||||
});
|
||||
|
||||
expect(cancelQueries).toHaveBeenCalledWith({
|
||||
queryKey: ["integrations", "lark"],
|
||||
});
|
||||
expect(queryClient.getQueryData(["integrations", "lark"])).toEqual(status);
|
||||
});
|
||||
});
|
||||
Loading…
x
Reference in New Issue
Block a user