mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 13:16:24 +00:00
2ad2132336
* fix(citations): pair inline code within a paragraph, not across blank lines maskInlineCode paired backticks over the whole document, so a stray backtick in one paragraph and another in a later one formed a span CommonMark never renders. The citation between them was masked out of the Sources list, and the stolen opener simultaneously un-masked a real code span further down. Scope the pairing to blank-line-delimited paragraphs, keeping the captured separator so string length and occurrence indices still align with the original markdown. * fix(citations): end the paragraph on a CRLF blank line too core/messages/utils.ts recognises a blank line as \r?\n[ \t]*\r?\n; the citations scanner now uses the same shape, so CRLF markdown gets the same pairing limit instead of falling back to whole-document pairing. * fix(citations): mask fences in containers and end inline spans at block bounds Follow-up to the paragraph-scoped inline pairing review on #5876. Indented fences inside a list item, and blockquote-prefixed fences, were only blanked by accident before: the whole-document backtick pairing happened to close them. Scoping the pairing to a paragraph removed that safety net and let sample citations surface as sources, so the fence matcher now accepts container prefixes and is marker-aware. Inline spans also end at the other block boundaries CommonMark uses - ATX headings, thematic breaks / setext underlines, interrupting list items and a blockquote's empty continuation line - not only at blank lines. * fix(citations): anchor fence detection and end fences with their container An unanchored container-prefix match turns any line that merely contains a three-backtick run into a fence opener, which masks everything after it and drops live citations. Anchor the match to the line start. A fence also has to end with the container it was opened in: a blockquote loses its fence at the first line without a `>` marker, and an indented fence at the first non-blank line that is less indented than its opener. Without that, the citation following a closed blockquote is masked away, which is the same failure direction as the original bug. * fix(citations): track the container a fence was opened in Separate a fence marker's own indentation from the indentation its container gives it, require whitespace after a list marker before treating it as a container, count blockquote depth instead of testing it as a boolean, and check a closer's indentation against the container before ending the block. Cover the four shapes from the review on #5876 with tests, and check the scanner against remark-parse over twenty fence shapes. * fix(citations): cap an opener's indentation at three columns past its container A marker four columns past the container's content column is an indented code block, so it must not open a fence; otherwise the citation below it is blanked to end of input instead of surfacing as a source. * fix(citations): match fence openers and closers to what their tails allow A closing fence is bare, and a backtick fence's info string carries no backtick. So ` ```text ` inside a ``` block is literal content rather than a closer, and ` ```md `x` ` is a paragraph line rather than an opener. Missing either check sends the scanner the wrong way: the first lets a citation after the fake closer surface as a phantom source, the second blanks a citation the reader can actually click. * fix(citations): measure fence columns in columns, and only close on a closer A tab after a list marker advances to the next four-column stop, so sizing a list item by the character length of its marker put the content column two columns too far left and kept a dedented citation masked inside the fence. A closing fence now comes from its own pattern instead of the opener's: the opener also accepts list markers, which are plain content once a block is open, so `- ``` ` inside a fence closed it early and surfaced the citation under it. The optional space after a block quote marker is padding, not indentation, so measuring a quoted marker from the raw prefix cost the fence one column of its three-column budget and rejected a valid closer. * test(citations): pin the list-nested closer shape as its own case * fix(citations): keep a blockquote fence escape inside an indented code run Losing the `>` closes a quoted fence, but the escaping line is not free text: four columns of indentation at the top level starts an indented code block, so its citations are still code. Without that, the paragraph-scoped inline pairing this scanner replaced leaves them visible as phantom sources. * fix(citations): read blank lines and surviving containers around quote escapes * fix(citations): count a quote that opens behind a list marker BLOCKQUOTE_PREFIX_RE matches from the start of a line, so a fence opened as "- > ```md" recorded quoteDepth 0 even though it sits inside a quote. The escape test then never fired and the fence masked to end of input, blanking citations that remark-parse renders as live links: 35 of the 924 swept container combinations were worse than main, all in that family. Read the opener's own container prefix instead, and park the list stack the quote hides in so a dedented line is measured against the item that survives it. * fix(citations): keep literal quote lines and multi-marker fences contained A fence that was not opened in a block quote has no quote container a later line can leave, so a `>` inside it is literal code: stripping it made an indented content line read as column zero and escape the fence. A line can also open several list items at once, and the fence belongs to the deepest of them, so recording only the first marker put `- - - ```md` four columns past its container and rejected a valid opener. * fix(citations): scan the container prefix in one pass Reading the containers a fence marker may sit behind was a single regex whose alternation could take a space either as a step of its own or as the padding of the next marker, so it described the same prefix exponentially many ways. A line carrying no fence therefore made the engine enumerate every decomposition: the deep-quote shapes tests/e2e/user-message-plain-text.spec.ts renders cost a second at 26 markers, roughly double for each marker added, and never return for the three thousand markers those messages carry, which is why the e2e job timed out waiting for a message that had already rendered on main. The scan now walks each position once. Which shapes open a fence is unchanged: the 32-shape and 35-shape differentials against remark-parse, the 924-combination sweep and this file's 50 existing cases all give the same answers as before. * fix(citations): keep the list a finished quote lived in, and let paragraph text block a non-1 ordered fence maskFencedCodeBlocks cleared the item stack on every blockquote-depth change, so a fence opened after a quote ended measured against the document root and hid a citation the reader had already left the item for. It also pushed any ordered marker as a container, so a `2. ~~~md` line that CommonMark keeps inside the preceding paragraph opened a fence and masked a rendered citation. * fix(citations): test the paragraph boundary on the CR-normalized line A CRLF document keeps its "\r" on every line after the split on "\n", and THEMATIC_BREAK_RE is anchored at the end of the line, so a setext underline "---\r" was not recognised as a leaf block. The paragraph it had already ended stayed live, the following "2. ```md" was read as paragraph text, its fence never opened, and the real closer was then re-read as a top-level opener that masked the rendered citation to EOF. Use the same CR-normalized line the position classification already uses, and pin the LF/CRLF pair plus the cross-depth quote shape against the parser tree. * fix(citations): keep list items and quote markers the reader keeps Three container rules in the fence scanner disagreed with remark-parse: - A dedented line of paragraph text is a lazy continuation, so it does not leave the list item. Popping the item there recorded the next fence at column zero and hid a rendered citation to end of input. - Padding after a list marker tops out at four columns: the fifth belongs to an indented code block inside an item whose content starts one column past the marker, not to a column-six container. - A `>` deeper than the quote markers a quoted fence already consumed is literal code. Stripping it made the content look dedented past the fence's column and ended the fence one line early, surfacing a citation the reader shows as code. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>