[PreTest] Technical proposal for the checksum check of the three-party dependencies #431

NishantBansal2003 · 2024-08-04T11:54:20Z

1. Does this PR affect any open issues?(Y/N) and add issue references :

N
Y

2. What is the scope of this PR (e.g. component or file name):

/docs/proposal

3. Provide a description of the PR(e.g. more details, effects, motivations or doc link):

4. Are there any breaking changes?(Y/N) and describe the breaking changes(e.g. more details, motivations or doc link):

N
Y

5. Are there test cases for these changes?(Y/N) select and add more details, references or doc links:

github-actions · 2024-08-04T11:54:34Z

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

NishantBansal2003 · 2024-08-04T11:57:09Z

I have read the CLA Document and I hereby sign the CLA

NishantBansal2003 · 2024-08-04T11:57:24Z

recheck

NishantBansal2003 · 2024-08-04T12:52:18Z

Hi @zong-zhe @Peefy,
I have completed the pre-test for the technical proposal on checksum checks for third-party dependencies. I have explored several package managers to understand the approach for implementing checksum verification for dependencies.
Please review my work and suggest any improvements.
Thanks!

coveralls · 2024-08-04T15:50:41Z

Pull Request Test Coverage Report for Build 11678614661

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0 changed or added relevant lines in 0 files are covered.
1672 unchanged lines in 18 files lost coverage.
Overall coverage increased (+0.8%) to 41.2%

Files with Coverage Reduction	New Missed Lines	%
pkg/git/getter.go	4	80.0%
pkg/client/pull.go	18	70.67%
pkg/opt/opt.go	25	42.47%
pkg/cmd/cmd_update.go	26	0.0%
pkg/git/git.go	30	50.98%
pkg/package/toml.go	31	79.31%
pkg/cmd/cmd_add.go	35	0.0%
pkg/api/kpm_run.go	42	51.18%
pkg/downloader/toml.go	53	0.0%
pkg/mvs/mvs.go	58	55.88%

Totals
Change from base Build 10383495256:	0.8%
Covered Lines:	3741
Relevant Lines:	9080

💛 - Coveralls

Peefy · 2024-08-05T10:08:26Z

cc @zong-zhe

zong-zhe · 2024-08-06T02:37:26Z

docs/proposal/kcl-checksum-verification-proposal.md

+## For the existing kpm dependencies, we need a solution that does not break compatibility for starting this feature
+We can have two methods to ensure backward compatibility:
+
+1. Checksum Registry/Database:


I think the overall direction of this section is in the right direction 👍, and in the next stage, you may need to provide more details to design the CheckSum Database for kpm.

At first, you can design what information to store in the Database and what it should look like in kpm and KCL packages. Refer to languages like cargo, go, etc. to see how they store this information and what information should be stored.

zong-zhe · 2024-08-06T02:40:28Z

docs/proposal/kcl-checksum-verification-proposal.md

+We can take some motivation from npm, where we create a `registry` that contains metadata for all the packages and dependencies known to KPM. This is similar to the Go checksum database, which has `checksum database` for all the modules known to Go.
+If we are not able to find the checksum entry in the lockfile, we can retrieve it from the registry or `checksum database`. We can also add a flag, similar to Go, allowing users to skip checking the registry or checksum database and download the package even if it might be corrupted.
+
+2. FLAG_NO_SUM_CHECK:


For this section, it is important to clarify when checksums verification are requested in kpm, and what impact they may have on the current kpm process.

Combined with the current kcl run, kcl mod update, kcl mod push, kcl mod pull , sorts out their current work flow and the effective position of checksums verification.

You can use a simple illustration as the example to show the workflow for kpm：

kcl run -> step 1 -> step 2 -> .......

And checksum verification will add as :

verify sum here | kcl run -> step 1 -> step 2 -> ...... | require sum here

Hey @zong-zhe, do we need checksum verification in kcl run? I navigated through the workflow of kcl run but could not find any place where it downloads or fetches modules from Git or OCI registry as part of its operation. Instead, it only fetches information from the local storage path of external packages.

Hi @NishantBansal2003 😄

kpm downloads OCI and GIT related content via Downloader: https://github.com/kcl-lang/kpm/blob/main/pkg/downloader/downloader.go

Thank you for the clarification. I apologize if my previous message was unclear. What I meant to convey is that when we trigger the kcl run command in the CLI, the functions that are executed do not seem to involve downloading content from Git or an OCI registry. Instead, they appear to only access information from the local storage path of external packages.

Additionally, I wanted to clarify whether the workflow example you provided was just an illustrative example or if we actually need to explore the possibility of incorporating checksum verification during the kcl run command. Should I investigate how checksum verification might be integrated into the kcl run process, or was that meant as a general example?

Could you please confirm if I’m understanding both aspects correctly?

NishantBansal2003 · 2024-08-07T16:47:35Z

Hey @zong-zhe,
My updated proposal is nearly complete, and I will commit the changes once this discussion is addressed.
Thank you for your patience, and I apologize for the delay.

Signed-off-by: Nishant Bansal <[email protected]>

NishantBansal2003 · 2024-08-08T05:11:14Z

Hey @zong-zhe,
I have pushed the changes based on the review. Could you please review them and let me know if I am heading in the right direction? Also, please address this doubt: link to discussion, so I can make the necessary changes to the proposal. Additionally, we need to explore more about Merkle trees, as I am not familiar with their implementation.
Thank you.

NishantBansal2003 · 2024-08-10T09:17:47Z

Hey @zong-zhe, please take a look when you have a moment. I'm happy to incorporate any changes if needed.

zong-zhe · 2024-08-12T02:37:59Z

docs/proposal/kcl-checksum-verification-proposal.md

+We can take some motivation from npm, where we create a `registry` that contains metadata for all the packages and dependencies known to KPM. This is similar to the Go checksum database, which has `checksum database` for all the modules known to Go.
+If we are not able to find the checksum entry in the lockfile, we can retrieve it from the registry or `checksum database`. We can also add a flag, similar to Go, allowing users to skip checking the registry or checksum database and download the package even if it might be corrupted.
+
+2. FLAG_NO_SUM_CHECK:


Hi @NishantBansal2003 😄

kpm downloads OCI and GIT related content via Downloader: https://github.com/kcl-lang/kpm/blob/main/pkg/downloader/downloader.go

zong-zhe · 2024-08-12T02:40:20Z

docs/proposal/kcl-checksum-verification-proposal.md

+(Instead of using merkle tree for checksum database)We can take motivation from Cargo - where we store `Kpm module records` in Git Repository and each commit to this repository is signed with GPG keys. When we fetch metadata of the module we will also fetch its signed commit, after that we can verify the GPG signature of the commit, This verification ensures that the commits were made by an authorized entity and that the commit data has not been altered.
+
+
+### Work flow of kpm process and the effective position of checksum verification


Try to design a separate module called Checker that can be used to check whether a KCL dependency is valid. This includes checking whether the dependency's name is appropriate, whether the version number matches the semantic version, and the checksum rather than simply making changes to an existing process. Independent modules can help us better test and manage.

Sure, I will try to design this. I will keep you updated about the progress.

Does KCL impose any naming conventions on dependencies? If not, how can we determine whether a dependency name is appropriate?
cc: @zong-zhe

Signed-off-by: Nishant Bansal <[email protected]>

NishantBansal2003 · 2024-08-14T07:02:24Z

@zong-zhe, I’ve added the design for the checker module. PTAL..

zong-zhe · 2024-08-15T02:55:37Z

docs/proposal/kcl-checksum-verification-checker-module-design.md

+	pkg "kcl-lang.io/kpm/pkg/package"
+)
+
+type Version struct {


Hi @NishantBansal2003 😄

My suggestion here is that a simple composition pattern in OOP might be better, no particularly complex structure is required, just a Checker abstract interface that provides a uniform function to check whether a dependency is valid, and then provides IdentChecker, VersionChecker and SumChecker check a KCL dependent name, version, and sum. Finally, These three Checkers are combined into one DepChecker, which is also an implementation struct of Checker.

Yes, adopting the composition pattern with a Checker interface would be better. It enhances clarity, making the code easier to maintain and extend. I’ll proceed with this approach. Thanks for the suggestion!

zong-zhe · 2024-08-15T03:01:49Z

docs/proposal/kcl-checksum-verification-checker-module-design.md

+}
+
+// parseVersion parses a semantic version string and returns its major, minor, and patch components.
+func parseVersion(v string) (p Version, ok bool) {


The content here does not need to be manually implemented, we can complete through the tripartite library, the specific can refer to here：

https://github.com/kcl-lang/kpm/blob/bea45753c66734d7f5ebe2a44fe544d14402db13/pkg/semver/semver.go#L19C23-L19C33

if the version is not semver, NewVersion will return an error.

Yes, if we can find a library that simplifies our implementation, that would be great. I’ll look into this and refer to the tripartite library as suggested. Thank you for pointing it out!

zong-zhe · 2024-08-15T03:06:49Z

docs/proposal/kcl-checksum-verification-checker-module-design.md

+}
+
+// HashLocalCache calculates the hash of all files in the specified local cache directory.
+func HashLocalCache(localCachePath string) (string, error) {


This method has been defined before:

kpm/pkg/utils/utils.go

Line 32 in bea4575

func HashDir(dir string) (string, error) {

I took inspiration from Go's checksum generation (hence the "h1:" format). Though the differences are subtle, here are the trade-offs and when each method is preferable:

HashLocalCache: Hashes both file content and name. Use when file names matter.

HashDir: Can ignore certain files (e.g., .git). Use when excluding non-essential files is necessary.

HashLocalCache: Slower due to sorting and name hashing. Use for higher accuracy with filenames and ordering.

HashDir: Faster and simpler. Use when performance is critical and file names/order don’t matter.

I think we can discuss how to move forward from here.

NishantBansal2003 · 2024-08-16T05:21:58Z

Hey @zong-zhe, made some modifications based on reviews, PTAL

NishantBansal2003 · 2024-08-16T05:28:23Z

Could you share some of the goals we have while implementing checksum verification, so I can explore possible solutions?
cc: @zong-zhe

zong-zhe · 2024-08-19T06:38:02Z

docs/proposal/kcl-checksum-verification-checker-module-design.md

+
+// Checker is the interface for all dependency checkers.
+type Checker interface {
+	Check(d pkg.Dependency, localCachePath string) error


I think it is better to use the KclPkg structure as a parameter, which is the interface for checking whether the package's properties are valid.

Are you suggesting that I should refactor the Checker interface to use KclPkg instead of pkg.Dependency? This would allow each checker to access all the relevant information about the package and its dependencies in a single structure. Specifically:

Using KclPkg to check the dependency name , version, and checksum for each dependency.

Ensuring that the SumChecker respects the NoSumCheck flag in KclPkg.

Would this approach align with the design goals, or do you recommend any adjustments?

Using KclPkg to check the dependency name , version, and checksum for each dependency.
Ensuring that the SumChecker respects the NoSumCheck flag in KclPkg.

Yes, you can try to sync these into the doc PR and provide more design.

zong-zhe · 2024-08-19T06:41:00Z

docs/proposal/kcl-checksum-verification-checker-module-design.md

+	if err != nil {
+		return fmt.Errorf("failed to calculate checksum: %w", err)
+	}
+	if d.Sum != gotSum {


Is it compared with the sum value calculated by the local file ? I remember it was to be compared with the sum stored remotely or sum local cache. The purpose of checking the checksum is to prevent security risks caused by malicious replacement of remote third-party dependencies.

Thank you for the feedback. Yes, the method calculates the checksum of the local file or cache.
The current implementation ensures that the local file matches the expected checksum stored in the dependency object (d.Sum). This object should ideally be populated with the checksum value retrieved from a trusted remote source or sum local cache to ensure we are validating the file against a reliable source.

Are you suggesting that d.Sum may be compromised, and that we should include the actual design for retrieving the checksum from the mod file or remotely? Previously, I thought that if this was the first time a package was downloaded, we would get the checksum from the remote source. If it was not the first time, we would check the checksum from the mod lock file, which we have already populated in d.Sum.

Hi @NishantBansal2003 😄

I don't understand what you want to express. My understanding is that the main function of this checksum is to determine whether the sum in the lock file is the same as the sum on the remote end, ensuring the security of the package. If you think there might be a different reason or a different usage, you might need to provide things like rust or go that they do with checksum.

Apologies for the previous approach. Earlier, I thought that d.Sum in KPM stores the trusted checksum for a dependency. However, after carefully reviewing the KPM code, I now understand that d.Sum retrieves the checksum from the OCI registry or calculates it using HashDir (which is to be stored in the lock file). This checksum needs to be compared with a remote trusted source to ensure the integrity of the dependency. Am I correct now?

kpm/pkg/client/client.go

Lines 1226 to 1236 in bea4575

dep.FromKclPkg(dpkg)

dep.Sum, err = c.AcquireDepSum(*dep)

if err != nil {

return nil, err

}

if dep.Sum == "" {

dep.Sum, err = utils.HashDir(localPath)

if err != nil {

return nil, err

}

}

Sure, now I think I have some clarity. I will update the design doc based on the reviews.

zong-zhe · 2024-08-19T07:01:23Z

Could you share some of the goals we have while implementing checksum verification, so I can explore possible solutions? cc: @zong-zhe

Hi @NishantBansal2003 😄

It is mainly divided into three parts:

Complete the Checker design doc, and then develop the Checker.
Use the Checker in the kpm and pass the flag to control whether the sum check function is enabled.
Adjust the existing three-party dependency and add the check sum.

Signed-off-by: Nishant Bansal <[email protected]>

NishantBansal2003 · 2024-08-19T14:53:59Z

Hi @zong-zhe,
I've made some modifications to the code. Could you please review it and let me know if I'm heading in the right direction? I’d like to proceed further with the implementation of getTrustedSum.
Thanks!

NishantBansal2003 · 2024-08-25T05:22:10Z

Hey @zong-zhe, please take a look when you have a moment.

zong-zhe · 2024-08-27T12:55:45Z

Hi @NishantBansal2003 😄

You can first propose a PR to implement Checker. As a solution doc, this PR will not be merged now, because there may be adjustments in the design of the solution in the future. After the completion of the whole work, we will merge this PR.

NishantBansal2003 · 2024-08-28T04:10:01Z

I’d like to proceed further with the implementation of getTrustedSum.

Hey @zong-zhe, since the implementation of getTrustedSum is incomplete, should I still submit the current code as a PR, considering that any modifications can be made in the future before merging?

NishantBansal2003 · 2024-08-28T05:41:36Z

PTAL @ #470, I think we can proceed with further discussion.

Signed-off-by: Nishant Bansal <[email protected]>

NishantBansal2003 · 2024-09-25T06:01:55Z

Hey @zong-zhe, I have added the design for integrating the checker into kpm's workflow, PTAL. Also, I need some help with an issue I encountered while testing the changes locally: https://cloud-native.slack.com/archives/C05TC96NWN8/p1727073915893189?thread_ts=1727069902.233579&cid=C05TC96NWN8

NishantBansal2003 · 2024-09-25T15:52:22Z

I need some help with an issue I encountered while testing the changes locally: https://cloud-native.slack.com/archives/C05TC96NWN8/p1727073915893189?thread_ts=1727069902.233579&cid=C05TC96NWN8

I understood the difference from -

kpm/pkg/opt/opt.go

Lines 209 to 267 in b4d396a

    
           func NewRegistryOptionsFrom(rawUrlorOciRef string, settings *settings.Settings) (*RegistryOptions, error) { 
        
           	parsedUrl, err := url.Parse(rawUrlorOciRef) 
        
           	if err != nil { 
        
           		return nil, err 
        
           	} 
        
           	// parse the options from the local file path 
        
           	if parsedUrl.Scheme == "" || parsedUrl.Scheme == constants.FileEntry { 
        
           		localOptions, err := NewLocalOptionsFromUrl(parsedUrl) 
        
           		if localOptions != nil && err == (*reporter.KpmEvent)(nil) { 
        
           			return &RegistryOptions{ 
        
           				Local: localOptions, 
        
           			}, nil 
        
           		} 
        
           	} 
        
           	// parse the options from the git url 
        
           	// https, http, git and ssh are supported 
        
           	if parsedUrl.Scheme == constants.GitScheme || 
        
           		parsedUrl.Scheme == constants.HttpScheme || 
        
           		parsedUrl.Scheme == constants.HttpsScheme || 
        
           		parsedUrl.Scheme == constants.SshScheme { 
        
           		gitOptions := NewGitOptionsFromUrl(parsedUrl) 
        
           		if gitOptions != nil { 
        
           			return &RegistryOptions{ 
        
           				Git: gitOptions, 
        
           			}, nil 
        
           		} 
        
           	} 
        
           	// parse the options from the oci url 
        
           	// oci is supported 
        
           	if parsedUrl.Scheme == constants.OciScheme || 
        
           		parsedUrl.Scheme == constants.HttpScheme || 
        
           		parsedUrl.Scheme == constants.HttpsScheme { 
        
           		ociOptions := NewOciOptionsFromUrl(parsedUrl) 
        
           		if ociOptions != nil { 
        
           			return &RegistryOptions{ 
        
           				Oci: ociOptions, 
        
           			}, nil 
        
           		} 
        
           	} 
        
           	// If all the url are invalid, try to parse the options from the oci ref. 
        
           	ociOptions, err := NewOciOptionsFromRef(rawUrlorOciRef, settings) 
        
           	if err != nil { 
        
           		return nil, err 
        
           	} 
        
           	if ociOptions != nil { 
        
           		return &RegistryOptions{ 
        
           			Registry: ociOptions, 
        
           		}, nil 
        
           	} 
        
           	return nil, fmt.Errorf("invalid dependencies source: %s", rawUrlorOciRef) 
        
           }

But I'm not sure how to handle the following case:
Currently, my implementation raises an error if dep.Source.Oci is nil. I want to ensure that we handle both scenarios correctly during integration. Since dep.Source.Registry.Oci gets filled when downloading using the OCI reference, but not dep.Source.Oci, should I adjust the integration code to also populate dep.Source.Oci when dep.Source.Registry.Oci is present? Or do you have a better approach to address this?

NishantBansal2003 · 2024-09-25T16:34:53Z

I also want to know some package example, like k8s:1.28, do not have a sum field in their manifest, resulting in a manifest like map[org.opencontainers.image.created:2023-09-28T07:52:58Z]. This leads to checksum failures in some other packages, such as istio, kwok etc. How should I fix this issue?
This issue actually reminds me of this:

3. Adjust the existing three-party dependency and add the check sum.

So, should I adjust the existing third-party dependency to include the checksum in its manifest? If so, how can I do that?

zong-zhe · 2024-09-26T02:52:52Z

docs/proposal/kcl-checksum-integration-design.md

@@ -0,0 +1,231 @@
+# Integrating Checker into the Existing KPM Workflow


Hi, @NishantBansal2003 😄

There are some detailed issues in your proposal that we need to discuss before proceeding with the subsequent implementation. You need to supplement some details in your research plan.

For example:

In functions that trigger re-downloads, will the verification process of third-party libraries be triggered, such as go build?

If the local third-party libraries are modified, will the verification process of the third-party libraries be triggered? What are the verification results? For instance, Golang allows modifications to third-party libraries during development. After I modify the third-party libraries locally, how does the checksum work in Golang?

Additionally, please research how the above tasks are handled in npm and helm.

I made some modifications based on the reviews. Please take a look and let me know if I am heading in the right direction. Thanks!

Hi @NishantBansal2003 😄

Currently, this plan is heading in the right direction, but it requires a few adjustments, specifically a clear division of tasks. We need to list each command in KPM that requires the integration of the checksum feature as sub-tasks.

Next, we need to prepare for the integration. We require a seamless upgrade process that does not render all existing third-party libraries unusable due to the addition of the checksum.

First, we need to know the specifics of the sums for all official third-party libraries. You may need to use some of the API methods currently provided by KPM to write a simple Go tool to tally which third-party libraries have sum fields and which do not. You can add a GitHub Action in this repository https://github.com/kcl-lang/modules to call this Go tool to generate a report. You can directly submit the PR of this Go tool to this repository https://github.com/kcl-lang/modules.

We need to list each command in KPM that requires the integration of the checksum feature as sub-tasks.

Already added the kcl commands that might need checksum integration and provided a rough approach through the code. If anything is missing or incorrect, let me know so I can fix it and propose an overall design later.

Signed-off-by: Nishant Bansal <[email protected]>

NishantBansal2003 · 2024-10-04T06:48:22Z

Hey @zong-zhe,
I have also added a rough code design for integrating the checksum checker into the KPM workflow for each KCL command. This implementation is based on my understanding, and I’m open to any modifications that could make the integration code more robust and isolated. Please take a look whenever you have time. Thanks!

NishantBansal2003 · 2024-10-10T10:11:56Z

Currently, I am considering overwriting the package artifact and updating the manifest with the given version using PushWithOciManifest in the KPM code. I will push the same package and updated manifest again and try to bypass the condition that returns an error if the package version already exists. I searched for the oras package to update only the manifest but could not find a solution. That's why I am thinking of this approach. Is this okay, or is there a better way that I should research?
Edit: Maybe we can use crane package, which I found in some discussions, as it is specifically designed for this use case.
cc: @zong-zhe

zong-zhe · 2024-10-11T02:30:50Z

Currently, I am considering overwriting the package artifact and updating the manifest with the given version using PushWithOciManifest in the KPM code. I will push the same package and updated manifest again and try to bypass the condition that returns an error if the package version already exists. I searched for the oras package to update only the manifest but could not find a solution. That's why I am thinking of this approach. Is this okay, or is there a better way that I should research? Edit: Maybe we can use crane package, which I found in some discussions, as it is specifically designed for this use case. cc: @zong-zhe

Hi 😄 oras provides this function used to adjust the manifest in https://oras.land/docs/commands/oras_manifest_push, you can use this function to adjust the manifest, you can look at the code implementation in oras, and then write the corresponding go program through the oras API, and write the corresponding test cases, making sure that you only adjust the sum field of the manifest.

NishantBansal2003 · 2024-11-02T08:23:40Z

Hey @zong-zhe, could you review this code design whenever you have time? Here is the link:
https://github.com/kcl-lang/kpm/pull/431/files#diff-1de6327653b8b200fc992a0aaec4debf824688fae1795edb5534cd2f93fd324c
This design integrates the checksum checker into the KPM workflow for each KCL command. Please suggest any changes for better implementation and integration. I will open another PR for this after successfully merging the following one:
kcl-lang/modules#243

Signed-off-by: Nishant Bansal <[email protected]>

NishantBansal2003 force-pushed the proposal/checksum_verification branch from 5052f6c to 8a7d63e Compare August 4, 2024 11:54

Peefy added a commit to kcl-lang/cla.db that referenced this pull request Aug 4, 2024

@NishantBansal2003 has signed the CLA in kcl-lang/kpm#431

7c3b59c

zong-zhe reviewed Aug 6, 2024

View reviewed changes

NishantBansal2003 added 2 commits August 8, 2024 10:36

Completed PreTest for Checksum Verification

f385852

Signed-off-by: Nishant Bansal <[email protected]>

Added checksum database design and workflow of kcl commands

8d68dbd

Signed-off-by: Nishant Bansal <[email protected]>

NishantBansal2003 force-pushed the proposal/checksum_verification branch from b015467 to 8d68dbd Compare August 8, 2024 05:06

NishantBansal2003 requested a review from zong-zhe August 9, 2024 16:29

zong-zhe reviewed Aug 12, 2024

View reviewed changes

Added Checker module design

7ac84b9

Signed-off-by: Nishant Bansal <[email protected]>

zong-zhe reviewed Aug 15, 2024

View reviewed changes

NishantBansal2003 requested a review from zong-zhe August 16, 2024 10:04

zong-zhe mentioned this pull request Aug 19, 2024

[Pretest]: added the proposal for checksum verification #441

Closed

11 tasks

zong-zhe reviewed Aug 19, 2024

View reviewed changes

Modified Checker module design based on reviews

045840b

Signed-off-by: Nishant Bansal <[email protected]>

NishantBansal2003 force-pushed the proposal/checksum_verification branch from b42cf76 to 045840b Compare August 19, 2024 14:51

NishantBansal2003 requested a review from zong-zhe August 19, 2024 14:54

Added design for integrating checker in kpm workflow

f0a76a0

Signed-off-by: Nishant Bansal <[email protected]>

zong-zhe reviewed Sep 26, 2024

View reviewed changes

Fixed issues in design of integrating checker in kpm

27fb310

Signed-off-by: Nishant Bansal <[email protected]>

NishantBansal2003 requested a review from zong-zhe September 27, 2024 10:08

Added code design for integrating checker into kpm

01df4a6

Signed-off-by: Nishant Bansal <[email protected]>

NishantBansal2003 mentioned this pull request Oct 12, 2024

feat: add checksum validation to KCL packages kcl-lang/modules#243

Open

1 task

fixes code design

e2e0a0b

Signed-off-by: Nishant Bansal <[email protected]>

		(Instead of using merkle tree for checksum database)We can take motivation from Cargo - where we store `Kpm module records` in Git Repository and each commit to this repository is signed with GPG keys. When we fetch metadata of the module we will also fetch its signed commit, after that we can verify the GPG signature of the commit, This verification ensures that the commits were made by an authorized entity and that the commit data has not been altered.


		### Work flow of kpm process and the effective position of checksum verification

	dep.FromKclPkg(dpkg)
	dep.Sum, err = c.AcquireDepSum(*dep)
	if err != nil {
	return nil, err
	}
	if dep.Sum == "" {
	dep.Sum, err = utils.HashDir(localPath)
	if err != nil {
	return nil, err
	}
	}

		@@ -0,0 +1,231 @@
		# Integrating Checker into the Existing KPM Workflow

[PreTest] Technical proposal for the checksum check of the three-party dependencies #431

Are you sure you want to change the base?

[PreTest] Technical proposal for the checksum check of the three-party dependencies #431

Conversation

NishantBansal2003 commented Aug 4, 2024

1. Does this PR affect any open issues?(Y/N) and add issue references :

2. What is the scope of this PR (e.g. component or file name):

3. Provide a description of the PR(e.g. more details, effects, motivations or doc link):

4. Are there any breaking changes?(Y/N) and describe the breaking changes(e.g. more details, motivations or doc link):

5. Are there test cases for these changes?(Y/N) select and add more details, references or doc links:

github-actions bot commented Aug 4, 2024 • edited Loading

NishantBansal2003 commented Aug 4, 2024

NishantBansal2003 commented Aug 4, 2024

NishantBansal2003 commented Aug 4, 2024

coveralls commented Aug 4, 2024 • edited Loading

Pull Request Test Coverage Report for Build 11678614661

Warning: This coverage report may be inaccurate.

Details

💛 - Coveralls

Peefy commented Aug 5, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NishantBansal2003 commented Aug 7, 2024

NishantBansal2003 commented Aug 8, 2024

NishantBansal2003 commented Aug 10, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NishantBansal2003 Aug 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NishantBansal2003 commented Aug 14, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NishantBansal2003 Aug 15, 2024 • edited Loading

Choose a reason for hiding this comment

NishantBansal2003 commented Aug 16, 2024

NishantBansal2003 commented Aug 16, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NishantBansal2003 Aug 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zong-zhe commented Aug 19, 2024

NishantBansal2003 commented Aug 19, 2024

NishantBansal2003 commented Aug 25, 2024

zong-zhe commented Aug 27, 2024

NishantBansal2003 commented Aug 28, 2024

NishantBansal2003 commented Aug 28, 2024

NishantBansal2003 commented Sep 25, 2024

NishantBansal2003 commented Sep 25, 2024

NishantBansal2003 commented Sep 25, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zong-zhe Sep 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NishantBansal2003 commented Oct 4, 2024

NishantBansal2003 commented Oct 10, 2024 • edited Loading

zong-zhe commented Oct 11, 2024

NishantBansal2003 commented Nov 2, 2024

github-actions bot commented Aug 4, 2024 •

edited

Loading

coveralls commented Aug 4, 2024 •

edited

Loading

NishantBansal2003 Aug 12, 2024 •

edited

Loading

NishantBansal2003 Aug 15, 2024 •

edited

Loading

NishantBansal2003 Aug 19, 2024 •

edited

Loading

zong-zhe Sep 29, 2024 •

edited

Loading

NishantBansal2003 commented Oct 10, 2024 •

edited

Loading