mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-11-15 07:41:20 +00:00
f39cdd6077
* fix(mcp): keep rotating OAuth refresh tokens across restarts and Gateway instances
MCP servers on the OAuth refresh_token grant lost their rotated refresh
token whenever a new process or manager started (restart, tool-cache
reset for non-task servers, every personal-connection run, a second
Gateway instance) and re-seeded with the consumed configured token,
getting invalid_grant for good.
- New table mcp_oauth_tokens (migration 0047_mcp_oauth_tokens), one row
per refresh_token connection keyed (scope, owner_user_id, server_key):
('deployment', '', name) or ('personal', owner, personal runtime
name). Access and refresh tokens are DEER_FLOW_CREDENTIALS_KEY
ciphertext; a config fingerprint (token_url, client_id, grant, scope,
audience, seed) makes a changed seed win.
- OAuthTokenManager re-reads the row under its process lock, adopts a
peer's valid token, and otherwise claims the refresh with a lease plus
a version compare-and-set (database clock), fetches outside any
transaction, writes under the lease and releases it on failure.
- The Gateway installs the store (sync engine via to_thread) around the
LangGraph runtime for sqlite/postgres; personal managers pass their
owner. client_credentials stays in process; an unloadable key falls
back to process memory; database errors fail the refresh.
- credentials_key_consumers lists refresh_token MCP OAuth servers, so a
declared multi-instance deployment needs the shared key.
Docs: MCP_SERVER.md, CONFIGURATION.md, Helm README, README, mcp and
migrations AGENTS.md, CHANGELOG (en/zh).
* docs(changelog): link the MCP OAuth token store entry to #7207
* fix(mcp): prune removed connections' OAuth token rows and ask owners to re-authorize on invalid_grant
Review follow-up (willem-bd):
- Both MCP config writers drop mcp_oauth_tokens rows of connections a
save removes (deployment: _commit_mcp_config_write; personal:
_mutate, keeping each connection's enabled/disabled runtime-name
variants so a toggle keeps its chain). Best effort; no startup sweep,
since a missing config file would look like every server removed.
- invalid_grant on a refresh_token request raises
McpOAuthReauthorizationRequired: the tool error names the server and
asks for a new refresh token, plus the shared-key hint when this
instance could not decrypt the stored chain.
- Annotate the gateway helpers' config as AppConfig.
* fix(mcp): renew the OAuth refresh lease, keep unstored rotations, and key access tokens by request parameters
Review follow-up (willem-bd, second round):
- HTTPX timeouts bound each I/O wait, not a whole response: the refresh
holder now renews its lease every third of REFRESH_LEASE_SECONDS while
the token request runs and abandons it after REFRESH_MAX_SECONDS (120 s);
waiters outlast a live holder. Regression uses a real loopback endpoint
that streams its body for three lease lengths.
- A refresh whose tokens could not be stored keeps them pending
(unsaved_base_version): they are written back before an older stored
token is adopted or refreshed with, retried every few seconds while the
access token is still valid, and dropped once a peer stored a newer
chain. A failed save releases the lease.
- New token_request_fingerprint (extra_token_params and response fields):
a stored access token is reused only under the same request parameters;
otherwise the stored refresh-token chain is refreshed.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
87 lines
3.5 KiB
Python
87 lines
3.5 KiB
Python
"""ORM model registration entry point.
|
|
|
|
Importing this module ensures all ORM models are registered with
|
|
``Base.metadata`` so Alembic autogenerate detects every table.
|
|
|
|
The actual ORM classes have moved to entity-specific subpackages:
|
|
- ``deerflow.persistence.thread_meta``
|
|
- ``deerflow.persistence.run``
|
|
- ``deerflow.persistence.feedback``
|
|
- ``deerflow.persistence.user``
|
|
|
|
``RunEventRow`` remains in ``deerflow.persistence.models.run_event`` because
|
|
its storage implementation lives in ``deerflow.runtime.events.store.db`` and
|
|
there is no matching entity directory.
|
|
"""
|
|
|
|
from deerflow.persistence.agents.model import AgentRow
|
|
from deerflow.persistence.channel_connections.model import (
|
|
ChannelConnectionRow,
|
|
ChannelConversationRow,
|
|
ChannelCredentialRow,
|
|
ChannelOAuthStateRow,
|
|
)
|
|
from deerflow.persistence.channel_leases.model import ChannelLeaseRow
|
|
from deerflow.persistence.channel_runtime_configs.model import ChannelRuntimeConfigRow
|
|
from deerflow.persistence.channel_thread_bindings.model import ChannelThreadBindingRow
|
|
from deerflow.persistence.feedback.model import FeedbackRow
|
|
from deerflow.persistence.login_throttle.model import LoginThrottleRow
|
|
from deerflow.persistence.managed_subagents.model import ManagedSubagentRow
|
|
from deerflow.persistence.mcp_oauth_tokens.model import McpOAuthTokenRow
|
|
from deerflow.persistence.mcp_tasks.model import McpTaskRow
|
|
from deerflow.persistence.models.run_event import RunEventRow, RunEventThreadSeqRow
|
|
from deerflow.persistence.notification_deliveries.model import NotificationDeliveryRow
|
|
from deerflow.persistence.personal_access_tokens.model import PersonalAccessTokenRow
|
|
from deerflow.persistence.projects.model import ProjectDocumentRow, ProjectRow
|
|
from deerflow.persistence.run.model import RunChangeClockRow, RunRow
|
|
from deerflow.persistence.scheduled_task_events.model import ScheduledTaskEventRow
|
|
from deerflow.persistence.scheduled_task_runs.model import ScheduledTaskRunRow
|
|
from deerflow.persistence.scheduled_tasks.model import ScheduledTaskRow
|
|
from deerflow.persistence.skill_mutations.model import SkillAssetRow, SkillOperationRow, SkillOwnerRow, SkillProposalRow
|
|
from deerflow.persistence.subagent_batches.model import SubagentBatchItemRow, SubagentBatchRow
|
|
from deerflow.persistence.thread_meta.model import ThreadMetaRow
|
|
from deerflow.persistence.thread_reads.model import ThreadReadMarkerRow, ThreadReadVersionRow
|
|
from deerflow.persistence.user.im_notifications import ImNotificationPreferenceRow
|
|
from deerflow.persistence.user.model import UserPreferenceRow, UserRow
|
|
from deerflow.persistence.webhook_delivery.model import WebhookDeliveryRow
|
|
|
|
__all__ = [
|
|
"AgentRow",
|
|
"ChannelConnectionRow",
|
|
"ChannelConversationRow",
|
|
"ChannelCredentialRow",
|
|
"ChannelLeaseRow",
|
|
"ChannelOAuthStateRow",
|
|
"ChannelRuntimeConfigRow",
|
|
"ChannelThreadBindingRow",
|
|
"FeedbackRow",
|
|
"LoginThrottleRow",
|
|
"McpOAuthTokenRow",
|
|
"McpTaskRow",
|
|
"ManagedSubagentRow",
|
|
"NotificationDeliveryRow",
|
|
"PersonalAccessTokenRow",
|
|
"ProjectDocumentRow",
|
|
"ProjectRow",
|
|
"RunEventRow",
|
|
"RunEventThreadSeqRow",
|
|
"RunChangeClockRow",
|
|
"RunRow",
|
|
"SkillAssetRow",
|
|
"SkillOperationRow",
|
|
"SkillOwnerRow",
|
|
"SkillProposalRow",
|
|
"ScheduledTaskEventRow",
|
|
"ScheduledTaskRow",
|
|
"ScheduledTaskRunRow",
|
|
"SubagentBatchRow",
|
|
"SubagentBatchItemRow",
|
|
"ThreadMetaRow",
|
|
"ThreadReadMarkerRow",
|
|
"ThreadReadVersionRow",
|
|
"UserPreferenceRow",
|
|
"ImNotificationPreferenceRow",
|
|
"UserRow",
|
|
"WebhookDeliveryRow",
|
|
]
|