Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Add lychee link checker with pre-commit #1366

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

jbampton
Copy link
Member

Clean up and improve the docs. Update the Makefile.

https://github.com/lycheeverse/lychee

"Fast, async, stream-based link checker written in Rust. Finds broken URLs and mail addresses inside Markdown, HTML, reStructuredText, websites and more!"

Did you read the Contributor Guide?

Is this PR related to a JIRA ticket?

  • No, this is a documentation update. The PR name follows the format [DOCS] my subject.

What changes were proposed in this PR?

Updated our hooks with pre-commit autoupdate

Added the lychee link checker.

fixes #1360

lychee seems to be more advanced than markdown-link-check.

In future we can add pre-commit inside Docker to allow contributors to run the pre-commit tests with Docker.

Since link checker tests can sometimes make a lot of noise and fail, I have separated the link checker from the other pre-commit checks.

How was this patch tested?

Ran Makefile command:

make checklinks

Ran pre-commit ( no link checker):

make check

This is my current output from running lychee locally

Screenshot from 2024-04-26 03-50-43

We have two broken links.

Did this PR include necessary documentation updates?

  • No, this PR does not affect any public API so no need to change the documentation.

Clean up and improve the docs. Update the Makefile.

https://github.com/lycheeverse/lychee

"Fast, async, stream-based link checker written in Rust. Finds broken URLs and mail addresses inside Markdown, HTML, reStructuredText, websites and more!"
@jbampton
Copy link
Member Author

After we fix the remaining broken links we can then choose going forwards to just check the links in modified files by running pre-commit run lychee or pre-commit run lychee --all-files to check all Markdown files with GitHub Actions.

The full check is taking 3 minutes so far.

docs/setup/compile.md Outdated Show resolved Hide resolved
docs/setup/compile.md Outdated Show resolved Hide resolved
"http://localhost:.*",
"https://whimsy.apache.org/officers/acreq.*",
"https://whimsy.apache.org/roster/pmc/sedona.*",
"https://www.datasyslab.net/",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @jiayuasu this datasyslab link is broken

Apache Sedona was formerly called GeoSpark, initiated by Arizona State University [Data Systems Lab](https://www.datasyslab.net/).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jbampton Can you replace it with this one: https://datasystemslab.engineering.asu.edu/

@jbampton jbampton self-assigned this Oct 29, 2024
@jbampton jbampton changed the title [DOCS] Add lychee link checker with pre-commit [CI] Add lychee link checker with pre-commit Nov 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add markdown-link-check with pre-commit
2 participants