Skip to content

fix(parser): enhance handling of custom tags and markdown syntax #1904

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
meet-student wants to merge 1 commit into
base: main
Choose a base branch
from
+230 −25
Open

fix(parser): enhance handling of custom tags and markdown syntax #1904

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 31 additions & 2 deletions packages/x-markdown/src/XMarkdown/__tests__/Parser.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,34 @@ describe('Parser', () => {
});

describe('protectCustomTagNewlines', () => {
it('should keep markdown syntax inside custom tag children as plain text', () => {
const parser = new Parser({
components: { custom: 'div' },
});
const content = '<custom>{"sales":[{"name":"电子产品[202](红)","value":52000}]}</custom>';
const result = parser.parse(content);
expect(result).toContain('电子产品[202](红)');
expect(result).not.toContain('<a href=');
});

it('should keep markdown syntax inside unclosed custom tag children as plain text', () => {
const parser = new Parser({
components: { custom: 'div' },
});
const content = '<custom>{"sales":[{"name":"电子产品[202](红)","value":52000';
const result = parser.parse(content);
expect(result).toContain('电子产品[202](红)');
expect(result).not.toContain('<a href=');
});

it('should not protect native HTML tags configured as components', () => {
const parser = new Parser({
components: { p: 'p' },
});
const result = parser.parse('<p>[Google](https://google.com)</p>');
expect(result).toContain('<p>[Google](https://google.com)</p>');
});

it('should protect newlines inside custom tags when protectCustomTagNewlines is true', () => {
const parser = new Parser({
protectCustomTagNewlines: true,
Expand All @@ -92,14 +120,15 @@ describe('Parser', () => {
expect(result).not.toMatch(/<CustomComponent>First line<\/p>\s*<p>Second line/);
});

it('should not protect newlines when protectCustomTagNewlines is false', () => {
it('should protect custom tag content when protectCustomTagNewlines is false', () => {
const parser = new Parser({
protectCustomTagNewlines: false,
components: { CustomComponent: 'div' },
});
const content = '<CustomComponent>First line\n\nSecond line</CustomComponent>';
const result = parser.parse(content);
expect(result).toContain('<p>');
expect(result).toContain('<CustomComponent>First line\n\nSecond line</CustomComponent>');
expect(result).not.toMatch(/<CustomComponent>First line<\/p>\s*<p>Second line/);
});
Comment thread
meet-student marked this conversation as resolved.

it('should work normally when protectCustomTagNewlines is true but no custom components', () => {
Expand Down
44 changes: 44 additions & 0 deletions packages/x-markdown/src/XMarkdown/__tests__/index.test.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,50 @@ describe('XMarkdown', () => {
expect((container.firstChild as HTMLElement)?.innerHTML).toBe(html);
});

it('passes custom component children as plain text without parsing markdown links', () => {
let receivedChildren: React.ReactNode;
const markdown =
'<custom>{"sales":[{"name":"电子产品[202](红)","value":52000}],"majorGroupName":"华南师范大学[202](汕尾校区)"}</custom>';

const { container } = render(
<XMarkdown
content={markdown}
components={{
custom: (props) => {
receivedChildren = props.children;
return <span data-testid="custom">{props.children}</span>;
},
}}
/>,
);

expect(receivedChildren).toBe(
'{"sales":[{"name":"电子产品[202](红)","value":52000}],"majorGroupName":"华南师范大学[202](汕尾校区)"}',
);
expect(container.querySelector('a')).not.toBeInTheDocument();
});

it('passes streaming custom component children as plain text before the tag is closed', () => {
let receivedChildren: React.ReactNode;
const markdown = '<custom>{"sales":[{"name":"电子产品[202](红)","value":52000';

const { container } = render(
<XMarkdown
content={markdown}
streaming={{ hasNextChunk: true }}
components={{
custom: (props) => {
receivedChildren = props.children;
return <span data-testid="custom">{props.children}</span>;
},
}}
/>,
);

expect(receivedChildren).toBe('{"sales":[{"name":"电子产品[202](红)","value":52000');
expect(container.querySelector('a')).not.toBeInTheDocument();
});

it('walkToken', () => {
const walkTokens = (token: Token) => {
if (token.type === 'heading') {
Expand Down
178 changes: 155 additions & 23 deletions packages/x-markdown/src/XMarkdown/core/Parser.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,121 @@ const escapeReplacements: Record<string, string> = {
};
const getEscapeReplacement = (ch: string) => escapeReplacements[ch];

const NATIVE_HTML_TAGS = new Set([
'a',
'abbr',
'address',
'area',
'article',
'aside',
'audio',
'b',
'base',
'bdi',
'bdo',
'blockquote',
'body',
'br',
'button',
'canvas',
'caption',
'cite',
'code',
'col',
'colgroup',
'data',
'datalist',
'dd',
'del',
'details',
'dfn',
'dialog',
'div',
'dl',
'dt',
'em',
'embed',
'fieldset',
'figcaption',
'figure',
'footer',
'form',
'h1',
'h2',
'h3',
'h4',
'h5',
'h6',
'head',
'header',
'hgroup',
'hr',
'html',
'i',
'iframe',
'img',
'input',
'ins',
'kbd',
'label',
'legend',
'li',
'link',
'main',
'map',
'mark',
'menu',
'meta',
'meter',
'nav',
'noscript',
'object',
'ol',
'optgroup',
'option',
'output',
'p',
'picture',
'pre',
'progress',
'q',
'rp',
'rt',
'ruby',
's',
'samp',
'script',
'search',
'section',
'select',
'slot',
'small',
'source',
'span',
'strong',
'style',
'sub',
'summary',
'sup',
'table',
'tbody',
'td',
'template',
'textarea',
'tfoot',
'th',
'thead',
'time',
'title',
'tr',
'track',
'u',
'ul',
'var',
'video',
'wbr',
]);

export function escapeHtml(html: string, encode?: boolean) {
if (encode) {
if (other.escapeTest.test(html)) {
Expand All @@ -54,6 +169,10 @@ const TAIL_MARKER = Symbol('tailMarker');
// Type for tokens that can be marked for tail injection
type MarkableToken = Token & { [TAIL_MARKER]?: boolean };

type CustomTagPlaceholder = {
protected: string;
};

class Parser {
options: ParserOptions;
markdownInstance: Marked;
Expand Down Expand Up @@ -173,7 +292,9 @@ class Parser {
placeholders: Map<string, string>;
} {
const placeholders = new Map<string, string>();
const customTagNames = Object.keys(this.options.components || {});
const customTagNames = Object.keys(this.options.components || {}).filter(
(name) => !NATIVE_HTML_TAGS.has(name.toLowerCase()),
);

if (customTagNames.length === 0) {
return { protected: content, placeholders };
Expand Down Expand Up @@ -223,6 +344,12 @@ class Parser {
const result: string[] = [];
let lastIndex = 0;

const createPlaceholder = ({ protected: protectedContent }: CustomTagPlaceholder) => {
const ph = `\u0000XMDPLACEHOLDER${placeholderIndex++}\u0000`;
placeholders.set(ph, protectedContent);
return ph;
};

for (const pos of positions) {
if (pos.type === 'open') {
// Self-closing tags don't have inner content
Expand All @@ -246,22 +373,31 @@ class Parser {
result.push(content.slice(lastIndex, startPos));
}

if (innerContent.includes('\n\n')) {
const protectedInner = innerContent.replace(/\n\n/g, () => {
const ph = `__X_MD_PLACEHOLDER_${placeholderIndex++}__`;
placeholders.set(ph, '\n\n');
return ph;
});
result.push(openTag + protectedInner + closeTag);
} else {
result.push(openTag + innerContent + closeTag);
}
result.push(
createPlaceholder({
protected: openTag + innerContent + closeTag,
}),
);

lastIndex = endPos;
}
}
}

if (stack.length > 0) {
const open = stack[0];
if (lastIndex < open.start) {
result.push(content.slice(lastIndex, open.start));
}
const unclosedContent = content.slice(open.start);
result.push(
createPlaceholder({
protected: unclosedContent,
}),
);
return { protected: result.join(''), placeholders };
}

if (lastIndex < content.length) {
result.push(content.slice(lastIndex));
}
Expand All @@ -273,10 +409,11 @@ class Parser {
if (placeholders.size === 0) {
return content;
}
return content.replace(
/__X_MD_PLACEHOLDER_\d+__/g,
(match) => placeholders.get(match) ?? match,
);
let restored = content;
placeholders.forEach((value, placeholder) => {
restored = restored.split(placeholder).join(value);
});
return restored;
Comment on lines +412 to +416
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot May 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The current implementation of restorePlaceholders is inefficient for large documents or many placeholders because it performs a full string traversal (split().join()) for every single placeholder. Since placeholders are unique and non-overlapping, using a single regex replacement is much more performant.

    return content.replace(/\u0000XMDPLACEHOLDER\d+\u0000/g, (match) => {
      return placeholders.get(match) ?? match;
    });

}

/**
Expand Down Expand Up @@ -337,14 +474,9 @@ class Parser {
// Set tail injection flag
this.injectTail = parseOptions?.injectTail ?? false;

// Protect custom tags if needed
if (this.options.protectCustomTagNewlines) {
const { protected: protectedContent, placeholders } = this.protectCustomTags(content);
const parsed = this.markdownInstance.parse(protectedContent) as string;
return this.restorePlaceholders(parsed, placeholders);
}

return this.markdownInstance.parse(content) as string;
const { protected: protectedContent, placeholders } = this.protectCustomTags(content);
const parsed = this.markdownInstance.parse(protectedContent) as string;
return this.restorePlaceholders(parsed, placeholders);
Comment on lines +477 to +479
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot May 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The protectCustomTagNewlines option is now effectively ignored as protectCustomTags is called unconditionally in the parse method. This represents a significant change in behavior: markdown syntax will no longer be parsed inside any custom components provided in the components prop.

If this is the intended new default behavior, the protectCustomTagNewlines prop should be deprecated or its documentation updated to reflect that it no longer controls this protection. Otherwise, the logic should respect the flag to allow users to opt-out of this behavior if they want markdown parsing within their custom components.

Comment thread
meet-student marked this conversation as resolved.
}
}

Expand Down
Loading