-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove markdown parsing from frontend #1952
base: main
Are you sure you want to change the base?
Conversation
[marked](https://github.com/markedjs/marked/) was used to parse markdown for each text node in the browser which was making things very slow. This commit moves markdown parsing to the backend using the `comrak` crate. Some cleanups may be required that removes the marked.js files completely from the codebase but it'll happen later once I get this changes reviewed.
I expect the tests to fail but I'm not too worried about that right now. I want either @amitu or @Arpita-Jaiswal (or both) to first review these changes carefully as I'm not too confident about this. |
@siddhantk232 how have you tested it so far? Is fastn.com, fifthtry, ide etc working with it? |
Can you also create some stress test / benchmark files, eg with 10K ftd.text nodes (with slightly different content; generated using Python) and do a before and after (backend and frontend timing) to see if it indeed made any improvements or is it a performance regression? |
I tested it with the ftd snippet mentioned in the PR description which worked fine. fastn.com does have some visual differences (ftd code is not highlighted and other code is not indented properly with this change). I didn't see any noticeable performance gains either: Same webpage but with current build of fastn that includes this change |
I don't think moving the code highlight part (prism.js) to the backend would make this any better. We should probably not go ahead with this for now. |
marked was used to parse markdown
for each text node in the browser which was making things very slow.
This commit moves markdown parsing to the backend using the
comrak
crate.
Some cleanups may be required that removes the marked.js files
completely from the codebase but it'll happen later once I get current
changes reviewed.
ftd
code that was used to test thisI couldn't see any visual changes before and after this commit. Attaching screenshots anyway:
(before this commit)
(after this commit)