AochenShen99 658c39ccf7
feat(skills): Add native SkillScan phase 1 for skills (#3033)
* Add phase 1 skill static scanning

* Rework SkillScan phase 1 as native scanner

* refactor(skillscan): align phase 1 with trimmed RFC contract

- SecurityFinding: 7 fields (rule_id, severity, file, line, message,
  remediation, evidence); category/analyzer derive from the rule_id
  prefix, confidence/column/fingerprint/metadata removed
- scan_archive_preflight()/scan_skill_dir() are pure functions: no
  ScanContext, no policy schema; CRITICAL-blocks is a code constant and
  skill_scan.enabled is applied by enforce_static_scan()/callers
- secret-* evidence is redacted before findings leave the scanner
- de-dup keys on (rule_id, file, line) so repeated occurrences keep
  distinct locations for agent self-correction
- cloud-metadata detection consolidated into network-cloud-metadata
- nested zip members get a one-level stdlib magic-byte peek; an
  executable member escalates package-nested-archive to CRITICAL
- install metadata sidecar removed (Phase 7 decides if it is needed)
- rule specs moved next to their analyzers; skillscan/rules/ removed
- tests updated + new anchors: redaction, dedup lines, nested-zip
  escalation, single cloud-metadata rule, bundled-skill zero-CRITICAL

* fix(skillscan): tighten reverse-shell/secret/archive scan rules from review

Address PR #3033 review feedback on the native SkillScan analyzers:

- Reverse-shell false positives: split shell detection by signal strength
  (/dev/tcp/, nc -e stay CRITICAL; bash -i, mkfifo -> new HIGH
  shell-reverse-shell-heuristic, warn->LLM). The Python check is now
  AST-anchored on real socket.socket/os.dup2/subprocess call sites instead
  of raw-text substring matching, so prose/docstrings no longer hard-block.
- Secret evidence: _redact_secret_evidence returns [redacted] with no secret
  bytes (was value[:6], which leaked 2 real token bytes past the prefix).
- Archive DoS: cap outer archive member count (_MAX_ARCHIVE_MEMBERS=4096);
  scan_archive_preflight early-aborts with a package-too-many-members CRITICAL
  finding (routes through the existing blocked->400 fail-closed path).
- shell-destructive-command: broaden the rm -rf matcher to sensitive system
  roots (/home, /usr, /*, --no-preserve-root /) while leaving safe subpaths
  unflagged.
- Dead code: collapse _decode_text_for_analysis to a single decode path and
  drop the unused _TEXT_SUFFIXES set and _has_text_shebang helper.
- local_skill_storage: document why the host_path branch keeps app_config
  possibly-None (lazy kill-switch resolution; avoids eager get_app_config in
  config-free environments such as CI).

Tests: new negative/positive coverage in test_skillscan_native.py. Full
backend suite 6616 passed, 26 skipped.
2026-07-07 21:44:28 +08:00
..

Documentation

This directory contains detailed documentation for the DeerFlow backend.

Document Description
ARCHITECTURE.md System architecture overview
API.md Complete API reference
AUTH_DESIGN.md User authentication, CSRF, and per-user isolation design
CONFIGURATION.md Configuration options
SETUP.md Quick setup guide

Feature Documentation

Document Description
STREAMING.md Token-level streaming design: Gateway vs DeerFlowClient paths, stream_mode semantics, per-id dedup
FILE_UPLOAD.md File upload functionality
PATH_EXAMPLES.md Path types and usage examples
SANDBOX_MEMORY_PROFILING.md Sandbox memory baseline and runtime comparison guide
summarization.md Context summarization feature
plan_mode_usage.md Plan mode with TodoList
AUTO_TITLE_GENERATION.md Automatic title generation

Development

Document Description
TODO.md Planned features and known issues

Getting Started

  1. New to DeerFlow? Start with SETUP.md for quick installation
  2. Configuring the system? See CONFIGURATION.md
  3. Understanding the architecture? Read ARCHITECTURE.md
  4. Building integrations? Check API.md for API reference

Document Organization

docs/
├── README.md                  # This file
├── ARCHITECTURE.md            # System architecture
├── API.md                     # API reference
├── AUTH_DESIGN.md             # User authentication and isolation design
├── CONFIGURATION.md           # Configuration guide
├── SETUP.md                   # Setup instructions
├── FILE_UPLOAD.md             # File upload feature
├── PATH_EXAMPLES.md           # Path usage examples
├── summarization.md           # Summarization feature
├── plan_mode_usage.md         # Plan mode feature
├── STREAMING.md               # Token-level streaming design
├── AUTO_TITLE_GENERATION.md   # Title generation
├── TITLE_GENERATION_IMPLEMENTATION.md  # Title implementation details
└── TODO.md                    # Roadmap and issues