mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 21:26:39 +00:00
19b7590333
* feat(typesafe): expose instructions/criteria overrides in the risk-gate eval The pre-enablement evaluation could not answer the question an operator actually asks when the gates fail: is the miss/false-block a model limit or a rubric limit? --instructions/--criteria-true/--criteria-false pass the provider's rubric overrides through, so threshold, max_state_chars and the rubric can be calibrated in one loop. First online run (28 labeled cases, jev-latest): defaults failed risky_misses_zero (cat ~/.aws/credentials at p=0.47) and system_false_block_within_target (two long safe calls locally refused over max_state_chars). With threshold 0.4, max_state_chars 8000 and a criteria.false clarifying that long-but-fully-shown workspace writes are local and reversible, every gate passed in two independent runs (0 missed risky, 0.0% false block; risky floor 0.48, safe ceiling 0.28). Network p95 passed at 505-620ms in three runs and failed once at 1023ms — re-measure in the target environment before enablement. * feat(typesafe): record the effective risk policy in the eval report A custom-rubric run was indistinguishable from a default-policy run in the recorded configuration, so its verdict could not be reproduced. Publish the provider's built-in defaults and write the resolved instructions and both criteria as report["policy"] alongside model and threshold. * feat(typesafe): mark the eval policy on stdout and pin flag passthrough Address the PR #6179 review notes: read the rubric-override flags by direct attribute access so a plumbing mistake fails loudly instead of silently evaluating the default policy, print policy=default|custom in the report header so a custom-rubric run is distinguishable on the console too, and pin the flag-to-provider passthrough with an offline provider-construction test.