Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't short-circuit checks (one issue can in some cases hide additional findings) #101

Open
0xjjoyy opened this issue Apr 13, 2020 · 2 comments
Labels
enhancement New feature or request

Comments

@0xjjoyy
Copy link

0xjjoyy commented Apr 13, 2020

Use Case
I have a IAM Policy with say invalid service name "servicedoesntexist", invalid action "iamactiondoesntexists" and invalid condition key "conditionkeydoesntexist".

Observed Output
Parliament returns on the first finding invalid service "UNKNOWN_PREFIX" and returns a single finding.

Desired Output
Parliament continues and identifies all three findings. This way we can catch all issues with the IAM Policy in the pipeline, rather than iteratively updating and deploying to the pipeline to find each error.

@0xdabbad00 0xdabbad00 added the enhancement New feature or request label Apr 13, 2020
@0xdabbad00
Copy link
Collaborator

This is a hard problem, but is an important one, because for example, if you mute the finding for UNKNOWN_PREFIX, you will think your policy is fine, because that finding has resulted in the other findings being hidden because Parliament can't proceed with other checks until that one is resolved.

I think in a generic sense, this a hard problem to deal with because right now, by the time I get to each check, I assume that everything else about the policy is basically correct. Breaking that assumption could make things a lot more complicated, but I think in specific cases, such as the issues indicated, it may be possible without too much complexity.

@dgubitosi
Copy link
Contributor

The UNKNOWN_* findings should not be muted, but they also should not abort the rest of the evaluation. They are not severe errors. They are no worse than a RESOURCE_MISMATCH. If changing this behavior may affect other projects, then maybe this should be an option.

@0xdabbad00 0xdabbad00 changed the title Identify all findings Don't short-circuit checks after one issue is found Jul 31, 2020
@0xdabbad00 0xdabbad00 changed the title Don't short-circuit checks after one issue is found Don't short-circuit checks (one issue can in some cases hide additional findings) Jul 31, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants