Initial CLI scanner commit

[aaronm-sysdig]

Initial PR for CLI v2 scanner functionality.

The main pain points here are the use of V2 api's due to the way that the v1beta apis do not carry the description and URL which we need (or if we don't we can cut it out... but its good to have them imho)

The methodology that I have taken is to try and be as respectful to the current logic as much as possible. So some things are a little longer than they would be had we have re-written everything from scratch. So for example rather than creating a new VulnerabilityItem construct for V2, it simply pulls the data from the V2 endpoints then transposes them into the v1 construct. This aides in allowing the logic to flow through the various other functions unchanged.

Points of Note that still need to be considered

• API's used are v2 api's as the current v1beta APIs for v2 functionality do not have (so far as I could tell) an API that would let you link together a scan result with an API that would let you query the description for a vulnerability like the v1 anchore API's do

• Descriptions are pulled from vulnPkgs endpoint. Sadly I have not found an API that allows us to get more than one description at once (i.e more than one image). This means that we hit up against the APi rate limit

• To the point of API rate limiting, I have modified the doRequest function to handle 429 - Too many requests and some rudimentary backoff functionality is implemented. Ideally we need to either 1) find a better API that gives all the descriptions and URLs we need OR have somewhere in the README that tells people to contact sysdig support to have their rate limit upgraded from 50 to xxx ?

• Vuln URLS come from the same description endpoint as could not find it in v1beta APi's.

• Logging has been extended retrospectively into the base adapter and logging of payloads is present in Debug mode.

• The API payload spec has been moved to 1.1, i.e "application/vnd.security.vulnerability.report; version=1.1. This allows us to present CVSS data into the report which was not present in 1.0

• APi spec as provided by Harbor is not correct. I had to scrape one from the trivvy scanner to see how to make it work (using vendor attributes

• Updates vendor tag from Sysdig 3.x to Sysdig SaaS.

• Readme has been updated to replace inline scanning to CLI scanning

• Command line parameter for scanning has changed from --inline_scanning to --cli_scanning

• Pod spec has changed to use a container that implements the CLI scanner. Currently just using a version that I wrote, but we will need to change this to sysdig hosted one. Dockerfile is below

FROM ubuntu:22.04@sha256:6d7b5d3317a71adb5e175640150e44b8b9a9401a7dd394f44840626aff9fa94d

RUN apt-get update && apt-get -y install curl

WORKDIR /root
RUN curl -LO "https://download.sysdig.com/scanning/bin/sysdig-cli-scanner/$(curl -L -s https://download.sysdig.com/scanning/sysdig-cli-scanner/latest_version.txt)/linux/amd64/sysdig-cli-scanner"
RUN chmod +x sysdig-cli-scanner

• CVSS Datai is now present with the 'tag' being Base_Score. This looks pretty good in the UI and is how other scanners present the data.

• Updated artifact tag stanza with image data from V2. Some transposing is needed so I have split it up into segments. Means more lines of code, but easier to read imho.

• Implemented REGISTRY_USER and REGISTRY_PASSWORD functionality that is (quietly) part of the CLI scanner to handle the robot account that is created for each scan so you can pull the image

• There is a bug in the UI at the moment that causes new vuln scans to now show. https://github.com/goharbor/harbor/issues/18523 is where I found it (took HOURS of beating head against a wall to work this out). The workaround is to clear the vuln tables for the scans in question. For debugging I just truncate the table with truncate scan_report, vulnerability_record cascade;. I wonder if the customer knows this??

[Jujuyeh]

Awesome work - LGTM