Skip to content

Commit

Permalink
Adds support for lowlevel GET calls against V3 API
Browse files Browse the repository at this point in the history
Before, one couldn't use the SnykClient to interface with the v3 endpoint. While long term V3 will have a new client, in the interim V3 doesn't have feature parity with V1, so for practical implementations a hybrid approach is required.

Because this update required substantially more test data to be used (multiple pages to test get_v3_pages), a new function to enable loading the requests response data directly from json stored in a test_data folder was added. This also makes Black and other code formatters happy.

Some housekeeping was required to update testing packages to latest versions to ensure they performed consistently.
  • Loading branch information
mrzarquon committed Mar 8, 2022
1 parent 7670f4b commit 46c3758
Show file tree
Hide file tree
Showing 14 changed files with 874 additions and 230 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
.coverage
.idea
.mypy_cache
.pytest_cache

__pycache__
*.pyc
Expand Down
127 changes: 91 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ client = snyk.SnykClient("<your-api-token>", tries=4, delay=1, backoff=2)
- `tries` - the maximum number of attempts. **Default:** `1` (no retries)
- `delay` - initial delay between attempts. **Default:** `1`
- `backoff` - multiplier applied to delay between attempts. **Default:** `2`
- `debug` - run client in debug mode, useful for debugging API requests. **Default:** `False`

## Organizations

Expand All @@ -60,21 +61,21 @@ Most of the API is scoped to organizations, so most other methods are found on t

The `snyk.models.Organization` object has the following properties related to the API:

* `entitlements` - returns the set of Snyk features available to this account
* `dependencies`- returns a Manager for packages in use in this organization
* `licenses` - returns a Manager for licenses currently in use by projects in this organisation
* `members` - returns a Manager for members
* `projects` - returns a Manager for associated projects
* `integrations` - returns a Manager for active integrations
- `entitlements` - returns the set of Snyk features available to this account
- `dependencies`- returns a Manager for packages in use in this organization
- `licenses` - returns a Manager for licenses currently in use by projects in this organisation
- `members` - returns a Manager for members
- `projects` - returns a Manager for associated projects
- `integrations` - returns a Manager for active integrations

### A note on Managers

Managers provide a consistent API for accessing objects from the Snyk API. Each manager implements the following methods:

* `all()` - return a list of all of the relevant objects
* `get("<id>")` - return a single instance of the object if it exists
* `first()` - grab the first instance of the object if one exists
* `filter(<key>="<value>")` - return a list filtered by one or more key/value pairs
- `all()` - return a list of all of the relevant objects
- `get("<id>")` - return a single instance of the object if it exists
- `first()` - grab the first instance of the object if one exists
- `filter(<key>="<value>")` - return a list filtered by one or more key/value pairs

### Projects

Expand All @@ -94,20 +95,20 @@ client.projects.all()

The `snyk.models.Project` object has the following useful properties and methods:

* `delete()` - deletes the project in question. Be careful as this will delete all associated data too
* `dependencies` - returns a Manager for packages in use in this project
* `dependency_graph` - returns a `snyk.models.DependencyGraph` object which represents the full dependency graph of package dependencies
* `ignores` - returns a Manager for ignore rules set on the project
* `vulnerabilities` - returns a list of `snyk.models.Vulnerability` objects with information about vulnerabilities in this project
* `jira_issues` - returns a Manager with access to any associated Jira issues
* `licenses` - returns a Manager for licenses currently in use by this project
* `settings` - returns a Manager for interacting with the current project settings
* `tags` - returns a Manager for interacting with the current project tags
- `delete()` - deletes the project in question. Be careful as this will delete all associated data too
- `dependencies` - returns a Manager for packages in use in this project
- `dependency_graph` - returns a `snyk.models.DependencyGraph` object which represents the full dependency graph of package dependencies
- `ignores` - returns a Manager for ignore rules set on the project
- `vulnerabilities` - returns a list of `snyk.models.Vulnerability` objects with information about vulnerabilities in this project
- `jira_issues` - returns a Manager with access to any associated Jira issues
- `licenses` - returns a Manager for licenses currently in use by this project
- `settings` - returns a Manager for interacting with the current project settings
- `tags` - returns a Manager for interacting with the current project tags

You can add and delete tags using the manager:

* `tags.add(key, value)` - adds a tag with the provided key/value pair to the project
* `tags.delete(key, value)` - deletes a tag with the provided key/value pair from the project
- `tags.add(key, value)` - adds a tag with the provided key/value pair to the project
- `tags.delete(key, value)` - deletes a tag with the provided key/value pair from the project

In the case of Projects, as well as filtering by properties (as mentioned above) you can also filter by tag:

Expand Down Expand Up @@ -139,16 +140,14 @@ org.import_project("github.com/user/project@branch", files=["Gemfile.lock"])

This method currently only supports importing projects from GitHub and Docker Hub. For other integrations you will need to grab the lower-level `snyk.models.Integration` object from the `snyk.models.Organization.integrations` manager noted above. Other services will be added to this API soon.


### Testing for vulnerabilities

The API also exposes methods to discover vulnerability information about individual packages. These methods are found on the Organization object.

* `test_maven(<package_group_id>, <package_artifact_id>, <version>)` - returns an IssueSet containing vulnerability information for a Maven artifact
* `test_rubygem(<name>, <version>)` - returns an IssueSet containing vulnerability information for a Ruby Gem
* `test_python(<name>, <version>)` - returns an IssueSet containing vulnerability information for Python package from PyPi
* `test_npm(<name>, <version>)` - returns an IssueSet containing vulnerability information for an NPM package

- `test_maven(<package_group_id>, <package_artifact_id>, <version>)` - returns an IssueSet containing vulnerability information for a Maven artifact
- `test_rubygem(<name>, <version>)` - returns an IssueSet containing vulnerability information for a Ruby Gem
- `test_python(<name>, <version>)` - returns an IssueSet containing vulnerability information for Python package from PyPi
- `test_npm(<name>, <version>)` - returns an IssueSet containing vulnerability information for an NPM package

Here's an example of checking a particular Python package.

Expand All @@ -166,14 +165,14 @@ False

As well as testing individual packages you can also test all packages found in various dependency management manifests. The client currently supports the following methods:

* `test_pipfile(<file-handle-or-string>)` - returns an IssueSet for all Python dependencies in a `Pipfile`
* `test_gemfilelock(<file-handle-or-string>)` - returns an IssueSet for all Ruby dependencies in a `Gemfile`
* `test_packagejson(<file-handle-or-string>, (<lock-file-handle-or-string>))` - returns an IssueSet for all Javascript dependencies in a `package.json` file. Optionally takes a `package.lock` file
* `test_gradlefile(<file-handle-or-string>)` - returns an IssueSet for all dependencies in a `Gradlefile`
* `test_sbt(<file-handle-or-string>)` - returns an IssueSet for all dependencies defined in a `.sbt` file
* `test_pom(<file-handle-or-string>)` - returns an IssueSet for all dependencies in a Maven `pom.xml` file
* `test_yarn(<file-handle-or-string>, <lock-file-handle-or-string>)` - returns an IssueSet for all dependencies in Yarn `package.json` and `yarn.lock` files
* `test_composer(<file-handle-or-string>, <lock-file-handle-or-string>)` - returns an IssueSet for all dependencies in Composer `composer.json` and `composer.lock` files
- `test_pipfile(<file-handle-or-string>)` - returns an IssueSet for all Python dependencies in a `Pipfile`
- `test_gemfilelock(<file-handle-or-string>)` - returns an IssueSet for all Ruby dependencies in a `Gemfile`
- `test_packagejson(<file-handle-or-string>, (<lock-file-handle-or-string>))` - returns an IssueSet for all Javascript dependencies in a `package.json` file. Optionally takes a `package.lock` file
- `test_gradlefile(<file-handle-or-string>)` - returns an IssueSet for all dependencies in a `Gradlefile`
- `test_sbt(<file-handle-or-string>)` - returns an IssueSet for all dependencies defined in a `.sbt` file
- `test_pom(<file-handle-or-string>)` - returns an IssueSet for all dependencies in a Maven `pom.xml` file
- `test_yarn(<file-handle-or-string>, <lock-file-handle-or-string>)` - returns an IssueSet for all dependencies in Yarn `package.json` and `yarn.lock` files
- `test_composer(<file-handle-or-string>, <lock-file-handle-or-string>)` - returns an IssueSet for all dependencies in Composer `composer.json` and `composer.lock` files

For example, here we are testing a Python `Pipfile`.

Expand All @@ -199,7 +198,6 @@ You can also invite new users as administrators:
>>> org.invite("[email protected]", admin=True)
```


### Low-level client

As well as the high-level API of the Snyk client you can use the HTTP methods directly. For these you simply need to pass the path, and optionally a data payload. The full domain, and the authentication details, are already provided by the client.
Expand All @@ -212,3 +210,60 @@ client.post("<path>", <data>)
```

Most of the time you shouldn't need to use these. They are mainly useful if new methods are added to the API which are not yet supported in the client. This can also be useful if you want to pass very specific parameters, or to parse the raw JSON output from the API.

## Experimental V3 Low Level Client

pysnyk >= 0.9.0 now includes support for basic V3 compatibility. To switch to use a V3 client, pass the V3 API url and version when initializing a client. Right now it supports the `GET` method. Refer to the [V3 API docs](https://apidocs.snyk.io/) for more information and examples.

Getting the V3 information of an organization:

```python

snyk_org = "39ddc762-b1b9-41ce-ab42-defbe4575bd6"

v3client = SnykClient(snyk_token,version="2022-02-16~experimental",url="https://api.snyk.io/v3")

print(v3client.get(f"/orgs/{snyk_org}").json())

# this supports overriding v3 versions for a specific GET requests:
user = v3client.get(f"orgs/{snyk_org}/users/{snyk_user}", version="2022-02-01~experimental").json()

# pass parameters such as how many results per page

params = {"limit": 10}

targets = v3client.get(f"orgs/{snyk_org}/targets", params=params)
```

V1 and V3 can work at the same time by instantiating two clients:

```python
snyk_org = "39ddc762-b1b9-41ce-ab42-defbe4575bd6"

v1client = SnykClient(snyk_token)

v3client = SnykClient(snyk_token,version="2022-02-16~experimental",url="https://api.snyk.io/v3")

v1_org = v1client.organizations.get(snyk_org)

v3_org = v3client.get(f"/orgs/{snyk_org}").json()
```

The V3 API introduces consistent pagination across all endpoints. The v3 client includes a helper method `.get_v3_pages` which collects the paginated responses and returns a single list combining the contents of the "data" key from all pages. It takes the same values as the get method.

```python
v3client = SnykClient(snyk_token,version="2022-02-16~experimental",url="https://api.snyk.io/v3")

params = {"limit": 10}

targets = v3client.get(f"orgs/{snyk_org}/targets", params=params).json()

print(len(targets["data"]))
# returns 10 targets

all_targets = v3client.get_v3_pages(f"orgs/{snyk_org}/targets", params=params)

print(len(all_targets))
# returns 33 targets, note we don't have to add .json() to the call or access the "data" key, get_v3_pages does that for us

```
Loading

0 comments on commit 46c3758

Please sign in to comment.