RuntimeError: memory access out of bounds with rehype

[SandroMaglione]

As reported before in #567 and #492, I am running in the same memory access out of bounds error.

The suggested fix (here and here) directly use getHighlighter/codeToHtml, but in my case I am using @shikijs/rehype, so I am not sure the same solution works.

The issue is hard to reproduce because it happens only when building on Vercel (locally it works) and only when building all the mdx files in my project (153 files, I tried with 127 files for example and it works).

This may also be related to a singleton issue with nextjs.

My setup is using next-mdx-remote (0.0.0-canary-20240321205249) and @shikijs/rehype (1.5.1), with a CustomMDX component as follows:

const rehypeShikiOptions = {
  theme: "one-dark-pro",
  langs: [
    /// ...
  ],
  transformers: [
    /// ...
  ],
} satisfies Partial<RehypeShikiOptions>;

const components = {
  /// ...
};

const options = {
  mdxOptions: {
    remarkPlugins: [remarkGfm],
    rehypePlugins: [
      [rehypeShiki, rehypeShikiOptions],
      rehypeSlug,
      rehypeAutolinkHeadings,
    ],
  },
} satisfies ComponentProps<typeof MDXRemote>["options"];

export function CustomMDX({ source }: { source: string }) {
  return (
    <MDXRemote
      source={source}
      components={components}
      options={options}
    />
  );
}

This is the full stack trace:

RuntimeError: memory access out of bounds\n
at wasm://wasm/001c7aca:wasm-function[103]:0x16c90\n
at wasm://wasm/001c7aca:wasm-function[198]:0x2509b\n
at OnigScanner._findNextMatchSync (file:///vercel/path0/node_modules/.pnpm/@[email protected]/node_modules/@shikijs/core/dist/index.mjs:4908:37)\n
at OnigScanner.findNextMatchSync (file:///vercel/path0/node_modules/.pnpm/@[email protected]/node_modules/@shikijs/core/dist/index.mjs:4900:21)\n
at CompiledRule.findNextMatchSync (file:///vercel/path0/node_modules/.pnpm/@[email protected]/node_modules/@shikijs/core/dist/textmate.mjs:1285:37)\n
at matchRule (file:///vercel/path0/node_modules/.pnpm/@[email protected]/node_modules/@shikijs/core/dist/textmate.mjs:2055:27)\n
at matchRuleOrInjections (file:///vercel/path0/node_modules/.pnpm/@[email protected]/node_modules/@shikijs/core/dist/textmate.mjs:2027:25)\n
at scanNext (file:///vercel/path0/node_modules/.pnpm/@[email protected]/node_modules/@shikijs/core/dist/textmate.mjs:1882:19)\n
at _tokenizeString (file:///vercel/path0/node_modules/.pnpm/@[email protected]/node_modules/@shikijs/core/dist/textmate.mjs:1878:9)\n
at Grammar._tokenize (file:///vercel/path0/node_modules/.pnpm/@[email protected]/node_modules/@shikijs/core/dist/textmate.mjs:2398:19)

Any idea on what can I try to fix the issue?