Merge branch 'canonical:main' into env-vars-extension

.github/workflows/publish.yaml

@@ -5,6 +5,8 @@ on:
   push:
     branches:
       - "feature/**"
+  # allow manual re-publishing as branches expire after 30 days
+  workflow_dispatch:
 
 jobs:
   publish:

.github/workflows/security-scan.yaml

@@ -0,0 +1,17 @@
+name: Security scan
+on:
+  pull_request:
+  push:
+    branches:
+      - main
+      - hotfix/*
+      - work/secscan  # For development
+
+jobs:
+  python-scans:
+    name: Scan Python project
+    uses: canonical/starflow/.github/workflows/scan-python.yaml@main
+    with:
+      packages: python-apt-dev
+      osv-extra-args: '--config=source/osv-scanner.toml'
+      trivy-extra-args: '--severity HIGH,CRITICAL --ignore-unfixed --skip-dirs "tests/spread/**"'

HACKING.md

@@ -98,8 +98,9 @@ tox run -e lint-codespell
 
 ## Evaluating pull requests
 
-Oftentimes all you want to do is see if a given pull request solves the issue you were having. To make this easier, the Travis CI setup for snapcraft _publishes_ the resulting snap that was built for x86-64 using `transfer.sh`.
-To download the snap, find the relevant CI job run for the PR under review and locate the "snap" stage, the URL to download from will be located at the end of logs for that job.
+Oftentimes all you want to do is see if a given pull request solves the issue you were having. To make this easier, a snap is published for `amd64` on a channel named `latest/edge/pr-<PR number>` where `PR number` is the number of the pull request.
+
+For feature branches, a snap is published for `amd64` on a channel named `latest/edge/<branch name>`. For example, a branch named `feature/offline-mode` would be available on the channel `latest/edge/offline-mode`.
 
 ## Reaching out
 

README.md

@@ -1,5 +1,4 @@
 [![snapcraft](https://snapcraft.io/snapcraft/badge.svg)](https://snapcraft.io/snapcraft)
-[![Build Status][travis-image]][travis-url]
 [![Documentation Status](https://readthedocs.com/projects/canonical-snapcraft/badge/?version=latest)](https://canonical-snapcraft.readthedocs-hosted.com/en/latest/?badge=latest)
 [![Scheduled spread tests](https://github.com/canonical/snapcraft/actions/workflows/spread-scheduled.yaml/badge.svg?branch=main)](https://github.com/canonical/snapcraft/actions/workflows/spread-scheduled.yaml)
 [![Coverage Status][codecov-image]][codecov-url]
@@ -29,8 +28,5 @@ Learn about the latest features by following Snapcraft on
 We love contributors. Read the [hacking guide](HACKING.md) if you're interested in helping out.
 
 
-[travis-image]: https://travis-ci.org/canonical/snapcraft.svg?branch=master
-[travis-url]: https://travis-ci.org/canonical/snapcraft
-
 [codecov-image]: https://codecov.io/github/canonical/snapcraft/coverage.svg?branch=master
 [codecov-url]: https://codecov.io/github/canonical/snapcraft?branch=master

TESTING.md

@@ -40,14 +40,6 @@ These tests are in the `tests/integration` directory, with the `snapcraft.yamls`
 At any time, an integration test may fail and given the use of temporary directories it can be hard to inspect what went on. When working on a specific test case you can set the environment variable `SNAPCRAFT_TEST_KEEP_DATA_PATH` to a directory path for the sepecic test.
 This mechanism will only work when working with individual tests and will fail to run with a batch of them.
 
-### Slow tests
-
-Some tests take too long. This affects the pull requests because we have to wait for a long time, and they will make Travis CI timeout because we have only 50 minutes per suite in there. The solution is to tag these tests as slow, and don't run them in all pull requests. These tests will only be run in autopkgtests.
-
-To mark a test case as slow, set the class attribute `slow_test = True`.
-
-To run all the tests, including the slow ones, set the environment variable `SNAPCRAFT_SLOW_TESTS=1`.
-
 ### Snaps tests
 
 The snaps tests is a suite of high-level tests that try to simulate real-world scenarios of a user interacting with snapcraft. They cover the call to snapcraft to generate a snap file from the source files of a fully functional project, the installation of the resulting snap, and the execution of the binaries and services of this snap.
@@ -120,35 +112,6 @@ We can currently run a minimal subset of snapcraft integration tests on macOS. T
 
 For manual exploratory testing, the team has one mac machine available.
 
-## Autopkgtests for the snapcraft deb
-
-Autopkgtests are tests for the project packaged as a deb. The unit tests are run during autopkgtests while the snapcraft deb is being built. Then the resulting deb is installed, and the integration and snaps suites are executed using the installed snapcraft.
-
-
-### How to run on Xenial
-
-The easiest way is to use a LXC container. From the root of the project, run:
-
-    sudo apt install autopkgtest
-    adt-run --unbuilt-tree . --apt-upgrade --- lxd ubuntu:xenial
-
-It's possible to select only one of the suites using `--testname`, for example:
-
-    adt-run --unbuilt-tree . --apt-upgrade --testname=integrationtests --- lxd ubuntu:xenial
-
-
-### How to run on Bionic
-
-The easiest way is to use a LXC container. From the root of the project, run:
-
-    sudo apt install autopkgtest
-    autopkgtest . -U -- lxd ubuntu:xenial
-
-It's possible to select only one of the suites using `--test-name`, for example:
-
-    autopkgtest . -U --test-name=integrationtests-spread -- lxd ubuntu:xenial
-
-
 ## Spread tests for the snapcraft snap
 
 [Spread](https://github.com/canonical/spread) is a system to distribute tests and execute them in different backends, in parallel. We are currently using spread only to run the integration suite using the installed snapcraft snap from the edge channel.

docs/reference/changelog.rst

@@ -1,3 +1,5 @@
+:tocdepth: 2
+
 Changelog
 *********
 

osv-scanner.toml

@@ -0,0 +1,4 @@
+[[IgnoredVulns]]
+id = "CVE-2024-35195"
+ignoreUntil = "2025-01-01T00:00:00Z"
+reason = "Needed for requests-unixsocket, which we're replacing with requests-unixsocket2"

requirements-devel.txt

@@ -199,5 +199,5 @@ yamllint==1.35.1
 zipp==3.20.2
 zope.deprecation==5.0
 zope.interface==7.0.3
-python-apt @ https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/python-apt/2.4.0ubuntu1/python-apt_2.4.0ubuntu1.tar.xz; sys.platform == "linux"
+python-apt @ https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/python-apt/2.4.0ubuntu1/python-apt_2.4.0ubuntu1.tar.xz ; sys.platform == "linux"
 pyinstaller==5.13.2; sys.platform == "win32"

requirements-docs.txt

@@ -147,4 +147,4 @@ websockets==12.0
 wheel==0.44.0
 ws4py==0.5.1
 zipp==3.20.2
-python-apt @ https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/python-apt/2.4.0ubuntu1/python-apt_2.4.0ubuntu1.tar.xz; sys.platform == "linux"
+python-apt @ https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/python-apt/2.4.0ubuntu1/python-apt_2.4.0ubuntu1.tar.xz ; sys.platform == "linux"

requirements.txt

@@ -78,4 +78,4 @@ wadllib==1.3.6
 wheel==0.44.0
 ws4py==0.5.1
 zipp==3.20.2
-python-apt @ https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/python-apt/2.4.0ubuntu1/python-apt_2.4.0ubuntu1.tar.xz; sys.platform == "linux"
+python-apt @ https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/python-apt/2.4.0ubuntu1/python-apt_2.4.0ubuntu1.tar.xz ; sys.platform == "linux"

snapcraft/commands/registries.py

@@ -82,6 +82,8 @@ class StoreEditRegistriesCommand(craft_application.commands.AppCommand):
 
         If the registries set does not exist, then a new registries set will be created.
 
+        If a key name is not provided, the default key is used.
+
         The account ID of the authenticated account can be determined with the
         ``snapcraft whoami`` command.
 
@@ -100,10 +102,14 @@ def fill_parser(self, parser: "argparse.ArgumentParser") -> None:
         parser.add_argument(
             "name", metavar="name", help="Name of the registries set to edit"
         )
+        parser.add_argument(
+            "--key-name", metavar="key-name", help="Key used to sign the registries set"
+        )
 
     @override
     def run(self, parsed_args: "argparse.Namespace"):
         self._services.registries.edit_assertion(
             name=parsed_args.name,
             account_id=parsed_args.account_id,
+            key_name=parsed_args.key_name,
         )

snapcraft/errors.py

@@ -173,3 +173,10 @@ def __init__(self, message: str, *, resolution: str) -> None:
             resolution=resolution,
             docs_url="https://snapcraft.io/docs/snapcraft-authentication",
         )
+
+
+class SnapcraftAssertionError(SnapcraftError):
+    """Error raised when an assertion (validation or registries set) is invalid.
+
+    Not to be confused with Python's built-in AssertionError.
+    """

snapcraft/models/assertions.py

@@ -16,13 +16,44 @@
 
 """Assertion models."""
 
-from typing import Literal
+import numbers
+from collections import abc
+from typing import Any, Literal
 
 import pydantic
 from craft_application import models
 from typing_extensions import Self
 
 
+def cast_dict_scalars_to_strings(data: dict) -> dict:
+    """Cast all scalars in a dictionary to strings.
+
+    Supported scalar types are str, bool, and numbers.
+    """
+    return {_to_string(key): _to_string(value) for key, value in data.items()}
+
+
+def _to_string(data: Any) -> Any:
+    """Recurse through nested dicts and lists and cast scalar values to strings.
+
+    Supported scalar types are str, bool, and numbers.
+    """
+    # check for a string first, as it is the most common scenario
+    if isinstance(data, str):
+        return data
+
+    if isinstance(data, abc.Mapping):
+        return {_to_string(key): _to_string(value) for key, value in data.items()}
+
+    if isinstance(data, abc.Collection):
+        return [_to_string(i) for i in data]
+
+    if isinstance(data, (numbers.Number, bool)):
+        return str(data)
+
+    return data
+
+
 class Registry(models.CraftBaseModel):
     """Access and data definitions for a specific facet of a snap or system."""
 
@@ -52,7 +83,6 @@ class EditableRegistryAssertion(models.CraftBaseModel):
     """Issuer of the registry assertion and owner of the signing key."""
 
     name: str
-    summary: str | None = None
     revision: int | None = 0
 
     views: dict[str, Rules]
@@ -61,6 +91,10 @@ class EditableRegistryAssertion(models.CraftBaseModel):
     body: str | None = None
     """A JSON schema that defines the storage structure."""
 
+    def marshal_scalars_as_strings(self) -> dict[str, Any]:
+        """Marshal the model where all scalars are represented as strings."""
+        return cast_dict_scalars_to_strings(self.marshal())
+
 
 class RegistryAssertion(EditableRegistryAssertion):
     """A full registries assertion containing editable and non-editable fields."""