Skip to content

Commit

Permalink
Feat trivy repo (#81)
Browse files Browse the repository at this point in the history
* added trivyRepoScan to api typedef and inputtypes

* added trivy repo scan to docs

* fixed indenting

* added trivy repo scan to index.js

* added trivy-repo vulnerability scan to all-checks.js

* cleaned up trivy install in Dockerfile, changed input_type and typedef in api, removed comment in gitleaks, strangely the mutation in index.js does not match the api, but still saving to database correctly, need to determine source of this.

* cleaned up comments
  • Loading branch information
LilaKelland authored Dec 13, 2023
1 parent 999cacb commit 4edbfd6
Show file tree
Hide file tree
Showing 9 changed files with 197 additions and 12 deletions.
3 changes: 2 additions & 1 deletion api/src/graphql_types/input_types.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ class GithubEndpointInput:
has_security_md: Optional[CheckPassesInput] = None
has_dependabot_yaml: Optional[CheckPassesInput] = None
gitleaks: Optional[CheckPassesInput] = None
hadolint: Optional[CheckPassesInput] = None
hadolint: Optional[CheckPassesInput] = None
trivy_repo_vulnerability: Optional[CheckPassesInput] = None


@strawberry.input
Expand Down
1 change: 1 addition & 0 deletions api/src/graphql_types/typedef.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ class GithubEndpoint(Endpoint):
has_dependabot_yaml: CheckPasses
gitleaks: CheckPasses
hadolint: CheckPasses
trivy_repo_vulnerability: CheckPasses

@strawberry.type
class Accessibility:
Expand Down
48 changes: 45 additions & 3 deletions docs/scanners.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,6 @@ Include a file called `Security.md` at the root of your repository explaining ho
"hasSecurityMd":{
"checkPasses": true,
"metadata": null,
"lastUpdated": 1698174245826
}
// ...
}
Expand Down Expand Up @@ -113,13 +112,13 @@ For preventative protection, consider using 'gitleaks protect' [pre-commit]((htt

> TODO
### Hadolint `Dockerfile`
### Hadolint Dockerfile Linting

[`Hadolint`](https://github.com/hadolint/hadolint) is a linter for Dockerfiles. This scanner analyzes the Dockerfiles in the source code repository, and flags any best practices rules that have been broken.

**Remediation**

Follow the guidelines outlined in results message to update the Dockerfiles. If your team has decided to not follow a particular rule in certain cases, you can clear the warning in this scanner by including an [inline ignore tag](https://github.com/hadolint/hadolint#inline-ignores) at the Dockerfile location where you would like to by-passed the rule check.
Follow the guidelines outlined in the results message to update the Dockerfiles. If your team has decided to not follow a particular rule in certain instances, you can clear the warning in this scanner by including an [inline ignore tag](https://github.com/hadolint/hadolint#inline-ignores) at the Dockerfile location where you would like to have the rule check by-passed.


**Data Example**
Expand Down Expand Up @@ -157,6 +156,47 @@ Follow the guidelines outlined in results message to update the Dockerfiles. If
}

```
### Trivy Repository Vunerability Scanning
[`Trivy`](https://github.com/aquasecurity/trivy) is a security scanner we're using in this case to scan software dependencies against known vunerabilities. It offers a remote Git repository scanner, that works for public repositories. Since we have some private repositories, we're using the filesystem scan on the cloned repository instead.
**Remediation**
Update the dependencies as indicated if there is a fixed version. Follow the URL for more information on the found vunerability.
**Data Example**
```jsonc
{
// ...
vulnerabilityTrivyRepoScan: {
checkPasses: false
metadata: [
{
library: "cryptography",
vulnerabilityID: "CVE-2023-49083",
severity: "MEDIUM",
installedVersion: "41.0.3",
fixedVersion: "41.0.6",
title: "cryptography is a package designed to expose cryptographic primitives ...",
url: "https://avd.aquasec.com/nvd/cve-2023-49083"
},
{
library: "cryptography",
vulnerabilityID: "GHSA-v8gr-m533-ghj9",
severity: "LOW",
installedVersion: "41.0.3",
fixedVersion: "41.0.4",
title: "Vulnerable OpenSSL included in cryptography wheels",
url: "https://github.com/advisories/GHSA-v8gr-m533-ghj9"
},
// ...
]
}
// ...
}

```
> TODO
Expand All @@ -168,3 +208,5 @@ Some products have one or more services exposed through URLs. URL compliance che
## Container Image Checks
Any products that build and deploy OCI images perform a series of checks on the built image artifact(s).
3 changes: 3 additions & 0 deletions scanners/github-cloned-repo-checks/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ RUN wget https://github.com/hadolint/hadolint/releases/download/v2.12.0/hadolint
mv hadolint-Linux-x86_64 /usr/local/bin/hadolint && \
chmod +x /usr/local/bin/hadolint

# Install trivy for repo scan
RUN curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin

COPY package*.json ./

RUN npm ci
Expand Down
14 changes: 10 additions & 4 deletions scanners/github-cloned-repo-checks/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,9 @@ process.on('SIGINT', () => process.exit(0))

// Mutation to add a graph for the new endpoints
// TODO: refactor this into a testable query builder function

// TODO: figure out why this only works with 'trivyRepoVulnerability', where it's coded as 'trivy_repo_vulnerability'
// in api, etc
const mutation = gql`
mutation {
githubEndpoint(
Expand All @@ -81,18 +84,21 @@ process.on('SIGINT', () => process.exit(0))
checkPasses: ${results.hadolint.checkPasses}
metadata: ${JSON.stringify(results.hadolint.metadata, null, 4).replace(/"([^"]+)":/g, '$1:')}
}
trivyRepoVulnerability: {
checkPasses: ${results.trivy_repo_vulnerability.checkPasses}
metadata: ${JSON.stringify(results.trivy_repo_vulnerability.metadata, null, 4).replace(/"([^"]+)":/g, '$1:')}
}
}
)
}
`;
}`;
console.log('*************************\n',mutation,'\n*************************\n')

// New GraphQL client - TODO: remove hard-coded URL
try {
const graphqlClient = new GraphQLClient(GRAPHQL_URL);

// Write mutation to GraphQL API
const mutationResponse = await graphqlClient.request(mutation);

console.log('Scan results saved to database.')

} catch (error) {
Expand All @@ -106,4 +112,4 @@ process.on('SIGINT', () => process.exit(0))

await nc.closed();

// nats pub "EventsScanner.githubEndpoints" "{\"endpoint\":\"https://github.com/PHACDataHub/ruok-service-autochecker\"}"
// nats pub "EventsScanner.githubEndpoints" "{\"endpoint\":\"https://github.com/PHACDataHub/ruok-service-autochecker\"}"
7 changes: 5 additions & 2 deletions scanners/github-cloned-repo-checks/src/all-checks.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { DotDockerIgnoreDetails, DotGitIgnoreDetails } from "./get-dotignore-de
import { CheckOnClonedRepoInterface } from './check-on-cloned-repo-interface.js'
import { Gitleaks } from './gitleaks.js'
import { Hadolint } from './hadolint.js'
import { TrivyRepo } from './trivy-repo.js'


export class AllChecks extends CheckOnClonedRepoInterface {
Expand All @@ -21,7 +22,8 @@ export class AllChecks extends CheckOnClonedRepoInterface {
new DotDockerIgnoreDetails(repoName, clonedRepoPath),
new DotGitIgnoreDetails(repoName, clonedRepoPath),
new Gitleaks(repoName, clonedRepoPath),
new Hadolint(repoName, clonedRepoPath)
new Hadolint(repoName, clonedRepoPath),
new TrivyRepo(repoName, clonedRepoPath)
];
}

Expand All @@ -41,7 +43,8 @@ export class AllChecks extends CheckOnClonedRepoInterface {
dotDockerIgnoreDetails: checkResults[4],
dotGitIgnoreDetails: checkResults[5],
gitleaks: checkResults[6],
hadolint: checkResults[7]
hadolint: checkResults[7],
trivy_repo_vulnerability: checkResults[8]
}

return allResults;
Expand Down
2 changes: 1 addition & 1 deletion scanners/github-cloned-repo-checks/src/gitleaks.js
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ async function runGitleaks(clonedRepoPath) {
// Remove temp dir
try {
await rm(tempDir, { recursive: true });
console.log(`Temporary directory ${tempDir} removed.`);
// console.log(`Temporary directory ${tempDir} removed.`);
} catch (removeError) {
console.error(`Error removing temporary directory ${tempDir}: ${removeError.message}`);
}
Expand Down
2 changes: 1 addition & 1 deletion scanners/github-cloned-repo-checks/src/hadolint.js
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ async function hadolintRepo(clonedRepoPath) {
results.push({
Dockerfile: relativePath,
RulesViolated: hadolintResult,
});
});
}

return results;
Expand Down
129 changes: 129 additions & 0 deletions scanners/github-cloned-repo-checks/src/trivy-repo.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@

// Trivy filesystem scan - looks for vunerablities based on lock files.
// https://aquasecurity.github.io/trivy/v0.17.2/scanning/filesystem/

// Note trivy has scan remote Git Repository functionality, but only available for public repositories. Since, we
// have the repositories cloned and available as filesystems, we will be using that option instead.

import { spawn } from 'child_process';
import path from 'path';
import { mkdtemp, rm } from 'fs/promises';
import os from 'os';
import { CheckOnClonedRepoInterface } from './check-on-cloned-repo-interface.js'
import fs from 'fs'


async function runTrivyScan(clonedRepoPath) {
try {
// create temp output file path
const tempDir = await mkdtemp(path.join(os.tmpdir(), 'trivy-'));
const scanResultsFilePath = path.join(tempDir, 'trivy-report.json');

// run trivy
const trivyProcess = spawn('trivy', [
'fs',
'-f', 'json',
'-o', scanResultsFilePath,
// '--scanners', 'vuln',
// '--scanners', 'misconfig',
// '--scanners', 'license', // need to npm install for this to work
clonedRepoPath
]);

trivyProcess.stderr.on('data', (data) => {
console.error(`Trivy stderr: ${data}`);
});

const code = await new Promise((resolve) => {
trivyProcess.on('close', resolve);
});

console.log(`Trivy process exited with code ${code}`);

if (code === 0) {
console.log(`Trivy scan completed successfully. Results saved to ${scanResultsFilePath}`);
const results = await parseScanResults(scanResultsFilePath);

// remove temp dir with results
await rm(tempDir, { recursive: true });
// console.log(`Temporary directory ${tempDir} removed.`);

return results;
} else {
console.error('Trivy scan failed.');
throw new Error('trivy failed');
}
} catch (error) {
console.error(`Error in runTrivyScan: ${error.message}`);
throw error;
}
}


async function parseScanResults(scanResultsFilePath){
// Reads trivy results from the json file, and pulls out relevant vunerability summary
let results = []
try {
const jsonData = fs.readFileSync(scanResultsFilePath, 'utf8');
const json = JSON.parse(jsonData);

// check for "Results" in data
if (json && json.Results && Array.isArray(json.Results)) {
json.Results.forEach(result => {
// Check if the "Vulnerabilities" in each result
if (result.Vulnerabilities && Array.isArray(result.Vulnerabilities)) {

result.Vulnerabilities.forEach(vulnerability => {
results.push({
library: vulnerability.PkgName,
vulnerabilityID: vulnerability.VulnerabilityID,
severity: vulnerability.Severity,
installedVersion: vulnerability.InstalledVersion,
fixedVersion: vulnerability.FixedVersion || 'N/A',
title: vulnerability.Title,
url: vulnerability.PrimaryURL
});

});
}
});
// console.log('*****************')
// console.log(results)
return results
}

} catch (error) {
console.error('Error reading or parsing JSON file:', error);
return []
}
}



export class TrivyRepo extends CheckOnClonedRepoInterface {
constructor(repoName, clonedRepoPath) {
super(repoName, clonedRepoPath);
this.clonedRepoPath = clonedRepoPath;
this.repoName = repoName;
}

async doRepoCheck() {
try {
const trivyRepoResult = await runTrivyScan(this.clonedRepoPath);

return {
checkPasses: (trivyRepoResult == 0),
metadata: trivyRepoResult
}

} catch (error) {
console.error(error.message);
return {
checkPasses: null,
metadata: {
errorMessage: 'An unexpected error occurred with trivy check.',
},
};
}
}
}

0 comments on commit 4edbfd6

Please sign in to comment.