mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-11-17 00:40:35 +00:00
* fix(skills): read legacy XLS workbooks with an XLS-capable engine * fix(skills): keep legacy XLS analysis independent of spatial * fix(skills): skip unreadable legacy workbooks * fix(skills): preserve mixed cell types in legacy XLS --------- Co-authored-by: xiaoyi <xyi74976-del@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2.5 KiB
2.5 KiB
Data analysis skill (skills/public/data-analysis/)
python -m pytest tests/skills -q -k "data_analysis"
Tests use real DuckDB. Default tests never download the spatial extension. XLSX
integration tests run only with an installed copy named by
DEERFLOW_TEST_SPATIAL_DIRECTORY. Legacy XLS and CSV analysis do not need it;
the XLS CLI regression uses an empty local extension repository in a fresh process.
Legacy XLS keeps compatible columns typed and stores incompatible mixes as text,
preserving nulls. Change the XLS cache key when this materialization changes.
Table names and SQL
- Lookup aliases and SQL table names are allocated together, for CSV files and Excel sheets. Neither may shadow an existing name, compared without case. Reserve a name only after its table was created.
- Bind file paths and sheet names as SQL values, and quote generated identifiers.
- User SQL is never rewritten with regular expressions. Original names resolve through connection-local temporary views over the canonical tables, also on a read-only cache hit. Normal CTE shadowing keeps working.
Cache
- Cache identity covers input order, file names and content digests. Moving the directory does not change it. Bump the namespace version when the table-mapping semantics change.
- Publish through
open_analysis_cache(). Build in a private directory, close the writer, reopen read-only, then atomically replace one manifest. Metadata is never published while a writer owns the database. - Validate the manifest and the physical tables before reuse. An invalid entry is rebuilt.
- Keep a generation when publication may have happened. Another process may be reading it. Concurrent cold builders can repeat work, and they never share a writable database.
- There is no automatic garbage collection and no power-loss guarantee.
Environment and exports
- Missing DuckDB or openpyxl prints setup instructions. The skill does not start an installer.
- Export paths resolve against the caller's working directory, also for a bare file name.
- Markdown export escapes headers and values the same way, and turns line
breaks into
<br>, so one result row stays one table row. - JSON export rejects duplicate column names before it opens the output file, and tells the caller to add SQL aliases. CSV and Markdown keep duplicates.
- At the CLI boundary a
ValueErrorbecomes a short stderr message and a non-zero exit. The connection closes infinally.