feat: add INVALID_REPOSITORY_VALUE rule

[Namchee]

Overview

Closes #97

This pull request adds new rule INVALID_REPOSITORY_VALUE that checks the validity of repository field in package.json.

How It Works

According to the specification, repository may be a string that represents a remote URL or an object with the following schema:

{
  "type": string;
  "url": string;
  "directory": string;
}

This rule will check whether the repository field is defined or not. If it's defined, the rule will perform regex matching if the value is a string or check both the url and directory property if it's an object.

Caveats

1. Currently, the URL regex doesn't support any remote URL other than GitHub.
2. type is not checked at the moment since the specification for that property is scarce.

[bluwy]

According to the specification, repository may be a string that represents a remote URL or an object with the following schema:

According to https://docs.npmjs.com/cli/v10/configuring-npm/package-json#repository, I'm not sure a remote URL string would work? At least when you run npm publish --dry-run, npm will try to convert it as an object instead

[Namchee]

According to the specification, repository may be a string that represents a remote URL or an object with the following schema:

According to https://docs.npmjs.com/cli/v10/configuring-npm/package-json#repository, I'm not sure a remote URL string would work? At least when you run npm publish --dry-run, npm will try to convert it as an object instead

Weird, it's used to work without a hitch (perhaps they enforced in it the newer version). Anyway, I'll put a suggestion to use object if the value is a string.

[bluwy]

Don't worry about the lint fail, I'm currently tweaking the changes locally and will push some fixes soon

[bluwy]

I've pushed some commits that:

1. Added docs
2. Simplified tests slightly
3. Updated how rules are reported and their messages

I think it should preserved the existing checks you did. Great addition for the git:// check and the shorthand regex check too. I still need to process the git url regex, but other than that I think it looks good to me now.

I'd appreciate if you can check the changes I made and if it works for you too.

[Namchee]

Thanks for taking your time on refactoring this PR! It looks much cleaner than I had expected, no issues from me.

As for the regex, I used these cases for testing, let me know if we missed something:

[email protected]:user/project.git
https://github.com/user/project.git
http://github.com/user/project.git
[email protected]:user/project.git
https://192.168.101.127/user/project.git
http://192.168.101.127/user/project.git
ssh://[email protected]:port/path/to/repo.git/
ssh://[email protected]/path/to/repo.git/
ssh://host.xz:1234/path/to/repo.git/
ssh://host.xz/path/to/repo.git/
ssh://[email protected]/path/to/repo.git/
ssh://host.xz/path/to/repo.git/
ssh://[email protected]/~user/path/to/repo.git/
ssh://host.xz/~user/path/to/repo.git/
ssh://[email protected]/~/path/to/repo.git
ssh://host.xz/~/path/to/repo.git
git://host.xz/path/to/repo.git/
git://host.xz/~user/path/to/repo.git/
http://host.xz/path/to/repo.git/
https://host.xz/path/to/repo.git/
git+https://github.com/user/repo.git
user@server:project.git
/path/to/repo.git/
path/to/repo.git/
~/path/to/repo.git
file:///path/to/repo.git/
file://~/path/to/repo.git/
host.xz:/path/to/repo.git/

Or you can play around on Debuggex (please enable multiline flag)

[bluwy]

Awesome. I pushed a commit that added some test for the utilities. Thanks for the debuggex link, IIUC the current regex doesn't seem to match all the valid git urls? I think that's fine as some valid git urls doesn't work for npm (example, edit, seems like ssh has a specific format (example)), but some valid ones aren't captured by the regex either it seems (example). Here's the github search I used to find these packages:

• https://github.com/search?q=repository+url+ssh+language%3Ajson+path%3A%2Fpackage.json%2F&type=code
• https://github.com/search?q=repository+url+gitlab.com+language%3Ajson+path%3A%2Fpackage.json%2F&type=code
• https://github.com/search?q=repository+%2F%22url%22%3A+%22ssh%3A%5C%2F%5C%2F%2F+language%3Ajson+path%3A%2Fpackage.json%2F&type=code

Maybe we should have this set of URLs to be matched by the regex only, a valid git URL that's also valid for npm. What do you think?

https://host.xz/path/to/repo.git/
http://host.xz/path/to/repo.git/
https://host.xz/path/to/repo.git
http://host.xz/path/to/repo.git
git+https://host.xz/path/to/repo.git
git+https://host.xz/path/to/repo
https://host.xz/path/to/repo.git
git+ssh://[email protected]/path/to/repo.git
git+ssh://[email protected]/path/to/repo
git://host.xz/path/to/repo.git
git://host.xz/path/to/repo

(Leaving out raw IP address and port as they're a bit fishy if used, but don't mind if the final regex can capture them either)

EDIT: I also found this fixture: https://github.com/npm/cli/blob/4e81a6a4106e4e125b0eefda042b75cfae0a5f23/test/lib/commands/repo.js#L115

[bluwy]

I also found https://github.com/npm/hosted-git-info, maybe we should simply use that 🤔

[bluwy]

I pushed a commit that updates the git URL regex so that while they may be valid git URLs per spec, they would also work for npm. The new regex also expands on the valid regexes that would previously be incorrectly marked some as invalid (e.g. ssh urls). I think this is good enough for now and I'm ready to not spend anymore time researching npm quirks 😅

Let me know if you have any more thoughts on this, otherwise I'll cut a release for this tonight.

[Namchee]

Personally, I would prefer if we can utilize hosted-git-info instead of trying to figure out the RegExp ourselves since the 'parser' is actually available.

Moreover, the RegExp seems to be unable to handle these cases correctly, hence strengthen the argument of using hosted-git-info instead.

I can work on this if you want

[bluwy]

Actually I forgot to comment about hosted-git-info. I tried to use it but the API feels awkward to use for our checks, for example (but I could be wrong), if repository is a string, we can't only validate that it's a valid shorthand. It'll treat http URLs as valid too. We also need to handle the github/gitlab suggestion ourselves because the package will autofix it.

It's probably easier to stick with the regex since these changes don't happen often, but if there's a neat way to reuse hosted-git-info without sacrificing the checks, that would be nice too.

Also, ssh://example.com:funny-port/repo is a valid name. For GitHub the funny-port is the username and it works on npm somehow.

[Namchee]

Well that sucks. Seems like the library is doing too much. At this point, I think it's better to resolve this and re-visit it later when someone created an issue here about the URL 😅. I swear npm has too many funny quirks.

[bluwy]

👍 Let's get this in first then. I just double checked how this change affects the popular packages listed on the site, and it seems to flag a lot of them with warnings. And strangely all those that we flagged still works on npm and displays the repository field properly. For now I'll mark all some of them as suggestions to prevent getting bashed and we can re-evaluate it again 😅