When using @docusaurus/faster, if there is some Chinese in the md file when packaging on version 3.6, an error message will be displayed, such as byte index 4 is not a char boundary.

[EaveLuo]

Have you read the Contributing Guidelines on issues?

• I have read the Contributing Guidelines on issues.

Prerequisites

• I'm using the latest version of Docusaurus.
• I have tried the npm run clear or yarn clear command.
• I have tried rm -rf node_modules yarn.lock package-lock.json and re-installing packages.
• I have tried creating a repro with https://new.docusaurus.io.
• I have read the console error message carefully (if applicable).

Description

● Client ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ (83%) sealing chunk ids
Panic occurred at runtime. Please file an issue on GitHub with the backtrace below: https://github.com/web-infra-dev/rspack/issues
Message: byte index 4 is not a char boundary; it is inside '点' (bytes 3..6) of 站点建设-7-fa-065
Location: crates\rspack_core\src\utils\compile_boolean_matcher.rs:325

Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
1: napi_register_module_v1
at
2: napi_register_module_v1
at
3: napi_register_module_v1
at
4: napi_register_module_v1
at
5: wasmer_vm_f64_nearest
at
6: wasmer_vm_f64_nearest
at
7: napi_register_module_v1
at
8: napi_register_module_v1
at
9: napi_register_module_v1
at
10: napi_register_module_v1
at
11: napi_register_module_v1
at
12: napi_register_module_v1
at
13: napi_register_module_v1
at
14: napi_register_module_v1
at
15: napi_register_module_v1
at
16: napi_register_module_v1
at
17: napi_register_module_v1
at
18: napi_register_module_v1
at
19: napi_register_module_v1
at
20: napi_register_module_v1
at
21: napi_register_module_v1
at
22: napi_register_module_v1
at
23: napi_register_module_v1
at
24: napi_register_module_v1
at
25: napi_register_module_v1
at
26: napi_register_module_v1
at
27: wasmer_vm_f64_nearest
at
28: napi_register_module_v1
at
29: napi_register_module_v1
at
30: napi_register_module_v1
at
31: BaseThreadInitThunk
at
32: RtlUserThreadStart
at
error Command failed with exit code 3221226505.

Reproducible demo

No response

Steps to reproduce

1.upgrade to 3.6.0
2.copy these to docusaurus.config.ts :
future: {
experimental_faster: {
swcJsLoader: true,
swcJsMinimizer: true,
swcHtmlMinimizer: true,
lightningCssMinimizer: true,
rspackBundler: true,
mdxCrossCompilerCache: true,
},
},
3.start or build
4.error happened

Expected behavior

package correctly and then start.

Actual behavior

I suspect there is a problem with the byte compilation of Chinese when packaging.

Your environment

• Public source code:https://github.com/EaveLuo/eave-web
• Public site URL:eaveluo.com
• Docusaurus version used:3.6.0
• Environment name and version (e.g. Chrome 89, Node.js 16.4):Edge Latest, Node.js 20.12.2
• Operating system and version (e.g. Ubuntu 20.04.2 LTS):windows10

Self-service

• I'd be willing to fix this bug myself.

[slorber]

Thanks for reporting.

I don't think it's related to Chinese because our Chinese site builds fine:
https://docusaurus.io/zh-CN/

What makes you think it's related to Chinese chars? Have you tried replacing those with non-Chinese chars and it fixed the issue?

Have you tried turning some of the Docusaurus Faster flags off one by one and see if it works better?

It looks like an Rspack bug, like the error message reports. cc @hardfist

[EaveLuo]

Thank you for reply.
I tried to delete the Chinese characters at the position where he reported the error, and then it would not report an error at this position, and then it would report errors about other Chinese characters, similar to next:

Panic occurred at runtime. Please file an issue on GitHub with the backtrace below: https://github.com/web-infra-dev/rspack/issues
Message: byte index 2 is not a char boundary; it is inside '后' (bytes 0..3) of 后端-403-60d
Location: crates\rspack_core\src\utils\compile_boolean_matcher.rs:325

[slorber]

I see thanks.

We also have @ruibaby that reported a similar issue, also involving Chinese characters apparently:
#10556 (comment)

@hardfist any idea if there's already a Rspack bug report for that?

I found a few possibly related links:

• fix(rspack_loader_runner): char_boundary panic when debug content web-infra-dev/rspack#6412
• Panic occurred at runtime. web-infra-dev/rspack#5085

[hardfist]

I see thanks.

We also have @ruibaby that reported a similar issue, also involving Chinese characters apparently: #10556 (comment)

@hardfist any idea if there's already a Rspack bug report for that?

I found a few possibly related links:

• fix(rspack_loader_runner): char_boundary panic when debug content web-infra-dev/rspack#6412
• Panic occurred at runtime. web-infra-dev/rspack#5085

thanks for reporting, we will take a look ASAP

[EaveLuo]

@slorber @hardfist In docusarus 3.6.2, enabling rspack for initial compilation now works fine, but the hot update error occurred.

Details:

PS D:\Code\eave-web> yarn start
yarn run v1.22.22
$ docusaurus start --host 0.0.0.0 --no-open --locale zh-CN
[INFO] Starting the development server...
[WARNING] Docusaurus site warnings:

• Your site is using the SWC js loader. You can safely remove the Babel config file at babel.config.js.
  [SUCCESS] Docusaurus website is running at: http://localhost:3000/
  ● Client ██████████████████████████████████████████████████ (100%) emitting after emit
  client (Rspack 1.1.2) compiled successfully
  ● Client ██████████████████████████████████████████████████ (77%) sealing chunk modules optimization
  Panic occurred at runtime. Please file an issue on GitHub with the backtrace below: https://github.com/web-infra-dev/rspack/issues
  Message: Chunk(ChunkUkey(Ukey(226), PhantomData<rspack_core::chunk::Chunk>)) not found in ChunkByUkey
  Location: crates\rspack_core\src\lib.rs:328

Backtrace omitted.

Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
error Command failed with exit code 3221226505.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

[JSerFeng]

is there a way i can repro this?

[EaveLuo]

is there a way i can repro this?有没有办法重现这个？

https://github.com/EaveLuo/eave-web/tree/renovate/docusaurus-monorepo

you can pull this repo, note the branch is /renovate/docusaurus-monorepo, then you can repro.

[ruibaby]

For me(halo-dev/docs), 3.6.2 has solved this problem, now pnpm start everything works fine

[slorber]

@EaveLuo is the new problem still related to Chinese characters?

[comp615]

@slorber FWIW, I'm also seeing the error mentioned by @EaveLuo, however I don't think we use Chinese characters? I don't quite know how to debug what we're looking at since the backtrace is not useful (non-public repo).

I did confirm that 3.6.1 does not have this issue (I can both start and do hot rebuilds)...3.6.2, initial build is OK, but all hot reloads crash the server

[hardfist]

@slorber is there any option do disable rspack incremental build in docusaurus now, the incremental is an experimental feature in rspack now, so it may contains bugs, so it maybe good to provide an option to disable incremental build in docusarus to avoid block users.

[EaveLuo]

@EaveLuo is the new problem still related to Chinese characters?

it shouldn't have anything to do with Chinese anymore, but rather with rspack.

[EaveLuo]

@slorber FWIW, I'm also seeing the error mentioned by @EaveLuo, however I don't think we use Chinese characters? I don't quite know how to debug what we're looking at since the backtrace is not useful (non-public repo).FWIW，我也看到了 提到的错误，但是_我认为我们没有_使用汉字吗？我不太知道如何调试我们正在查看的内容，因为回溯没有用（非公共 repo）。

I did confirm that 3.6.1 does not have this issue (I can both start and do hot rebuilds)...3.6.2, initial build is OK, but all hot reloads crash the server我确实确认了 3.6.1 没有这个问题（我既可以启动又可以进行热重建）......3.6.2，初始构建正常，但所有热重载都会导致服务器崩溃

I just enabled the entire project and it can be enabled normally, but the hot update will report an error, and directly enabling the English version will directly prevent it from starting.

[slorber]

Unfortunately I'm not able to reproduce the issue.

If you can, please try doing the following local modification and tell me if you can still see the bug?

node_modules/@docusaurus/core/lib/webpack/base.js

    function getExperiments() {
        if (props.currentBundler.name === 'rspack') {
            return {
-                incremental: !isProd,
+                incremental: undefined,
            };
        }
        return undefined;
    }

If that doesn't work, also try false instead of undefined

---

@slorber is there any option do disable rspack incremental build in docusaurus now, the incremental is an experimental feature in rspack now, so it may contains bugs, so it maybe good to provide an option to disable incremental build in docusarus to avoid block users.

We don't have any option but I could add one.

Since Rspack support is already experimental and opt-in, I thought about being a bit aggressive and using incremental: true in dev by default, considering users can still opt-out and use Webpack if really needed, and that we'll be able to help the Rspack team dogfood the feature and get potential bug reports like this to stabilize it.

In the short term, I think it's safer to disable it in the current minor and add a flag in next minor.

I wonder what should be the default flag value though, and how many users encounter this bug compared to those who do not encounter it. Any opinion?

[lebalz]

In my opinion a default flag value of false with a documented option of true might be better. Maybe not, since dx is not ideal without 😅. Currently, the compiler fails whenever a new file needs to be tracked...
The issue above was/is probably not the only one? (see web-infra-dev/rspack#8480 )

[comp615]

@slorber I tried your suggestion above. undefined appears to work for me. FWIW, false also works

Also updating RSPack seemed to fix this as referenced above: pnpm update @rspack/core

EDIT: I'm being told by devs it's still not fixed. So it works sometimes now, but not all

[slorber]

Going to disable this incremental feature in #10712 for the next patch release

We'll probably add a flag for it in the next major version so that users can opt-in/out of it until it works better.

I'm also closing this issue since the original Rspack bug has been fixed. This one is another thing and also likely an Rspack incremental problem.

[slorber]

Also updating RSPack seemed to fix this as referenced above: pnpm update @rspack/core

EDIT: I'm being told by devs it's still not fixed. So it works sometimes now, but not all

Are you sure 100% the dev was using Rspack 1.1.3? It seems it has only been released yesterday and includes a fix: https://github.com/web-infra-dev/rspack/releases/tag/v1.1.3

[comp615]

Also updating RSPack seemed to fix this as referenced above: pnpm update @rspack/core
EDIT: I'm being told by devs it's still not fixed. So it works sometimes now, but not all

Are you sure 100% the dev was using Rspack 1.1.3? It seems it has only been released yesterday and includes a fix: https://github.com/web-infra-dev/rspack/releases/tag/v1.1.3

I'm not, but it seems very likely since they saw some improvement and change in behavior when I updated the package lock to 1.1.3. But they also still had incremental: !isProd if that matters.

[slorber]

incremental: undefined is published in v3.6.3: https://github.com/facebook/docusaurus/releases/tag/v3.6.3

[slorber]

@comp615 thanks for the feedback.

That could probably be helpful for the Rspack team if you provided a bug repro, because it seems @ahabhgk latest incremental fix wasn't enough to make HMR work for your site.

[hardfist]

@slorber sorry for late reply

We don't have any option but I could add one.

Since Rspack support is already experimental and opt-in, I thought about being a bit aggressive and using incremental: true in dev by default, considering users can still opt-out and use Webpack if really needed, and that we'll be able to help the Rspack team dogfood the feature and get potential bug reports like this to stabilize it.

In the short term, I think it's safer to disable it in the current minor and add a flag in next minor.

I wonder what should be the default flag value though, and how many users encounter this bug compared to those who do not encounter it. Any opinion?

incremental flags is still in early experimental stage, we may consider enable it by default for rspress users first when we think it's stable enough, and when it's stable in rspress and then progressively enable it by default in rsbuild and rspress.
we don't have stats how many users encounter this bug compared to users enable it cause we don't do telemetry in rspack now.