Backports (stable-5.21) (#13857)

canonical · Aug 2, 2024 · ca4f14d · ca4f14d
2 parents b9d60b6 + 5ed6614
commit ca4f14d
Show file tree

Hide file tree

Showing 148 changed files with 8,507 additions and 2,719 deletions.
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
@@ -160,7 +160,7 @@ jobs:
         run: |
           set -eux
           sudo chmod o+w ./lxd/metadata/configuration.json
-          sudo chmod o+w ./doc/config_options.txt
+          sudo chmod o+w ./doc/metadata.txt
           sudo chmod o+w ./po/*
           make static-analysis
 

diff --git a/Makefile b/Makefile
@@ -90,6 +90,8 @@ ifneq "$(LXD_OFFLINE)" ""
 	exit 1
 endif
 	go get -t -v -d -u ./...
+	go get github.com/gorilla/[email protected] # Due to riscv64 crashes in LP
+	go get github.com/openfga/[email protected] # Due to build failures
 	go mod tidy -go=$(GOMIN)
 
 	@echo "Dependencies updated"
@@ -118,7 +120,7 @@ endif
 .PHONY: update-metadata
 update-metadata: build
 	@echo "Generating golang documentation metadata"
-	$(GOPATH)/bin/lxd-metadata . --json ./lxd/metadata/configuration.json --txt ./doc/config_options.txt --substitution-db ./doc/substitutions.yaml
+	$(GOPATH)/bin/lxd-metadata . --json ./lxd/metadata/configuration.json --txt ./doc/metadata.txt --substitution-db ./doc/substitutions.yaml
 
 .PHONY: doc
 doc: doc-clean doc-install doc-html doc-objects

diff --git a/doc/.sphinx/.markdownlint/exceptions.txt b/doc/.sphinx/.markdownlint/exceptions.txt
@@ -1,5 +1,5 @@
-.tmp/doc/howto/import_machines_to_instances.md:103: MD034 Bare URL used
-.tmp/doc/howto/import_machines_to_instances.md:207: MD034 Bare URL used
+.tmp/doc/howto/import_machines_to_instances.md:114: MD034 Bare URL used
+.tmp/doc/howto/import_machines_to_instances.md:218: MD034 Bare URL used
 .tmp/doc/howto/network_forwards.md:66: MD004 Unordered list style
 .tmp/doc/howto/network_forwards.md:71: MD004 Unordered list style
 .tmp/doc/howto/network_forwards.md:67: MD005 Inconsistent indentation for list items at the same level

diff --git a/doc/.wordlist.txt b/doc/.wordlist.txt
@@ -39,6 +39,7 @@ OEM
 OLM
 Permalink
 pre
+QCow
 Quickstart
 ReadMe
 reST
@@ -50,5 +51,9 @@ subtree
 Ubuntu
 UI
 UUID
+VDI
+VHDX
+VIRTIO
 VM
+VMDK
 YAML
diff --git a/doc/api-extensions.md b/doc/api-extensions.md
@@ -2424,3 +2424,14 @@ The OVN driver will allocate IP addresses from the subnets specified in the upli
 
 Adds the ability to explicitly specify a trust token when creating a certificate
 and joining an existing cluster.
+
+## `instance_import_conversion`
+
+Adds the ability to convert images from different formats (e.g. VMDK or QCow2) into RAW image format and import them as LXD instances.
+
+## `instance_create_start`
+
+Adds a `start` field to the `POST /1.0/instances` API which when set
+to `true` will have the instance automatically start upon creation.
+
+In this scenario, the creation and startup is part of a single background operation.
diff --git a/doc/contributing.md b/doc/contributing.md
@@ -147,15 +147,15 @@ Look for comments that start with `lxdmeta:generate` in the code.
 When you add or change a configuration option, make sure to include the required documentation comment for it.
 See the [`lxd-metadata` README file](https://github.com/canonical/lxd/blob/main/lxd/lxd-metadata/README.md) for information about the format.
 
-Then run `make generate-config` to re-generate the `doc/config_options.txt` file.
+Then run `make generate-config` to re-generate the `doc/metadata.txt` file.
 The updated file should be checked in.
 
-The documentation includes sections from the `doc/config_options.txt` to display a group of configuration options.
+The documentation includes sections from the `doc/metadata.txt` to display a group of configuration options.
 For example, to include the core server options:
 
 ````
-% Include content from [config_options.txt](config_options.txt)
-```{include} config_options.txt
+% Include content from [metadata.txt](metadata.txt)
+```{include} metadata.txt
     :start-after: <!-- config group server-core start -->
     :end-before: <!-- config group server-core end -->
 ```

diff --git a/doc/explanation/authorization.md b/doc/explanation/authorization.md
@@ -67,6 +67,8 @@ Some useful permissions at a glance:
 - The `user` entitlement on entity type `instance` grants access to view an instance, pull/push files, get a console, and begin a terminal session.
   Members of a group with this entitlement cannot edit the instance configuration.
 
+For a full list, see {ref}`permissions-reference`.
+
 ```{note}
 Due to a limitation in the LXD client, if `can_exec` is granted to a group for a particular instance, members of the group will not be able to start a terminal session unless `can_view_events` is additionally granted for the parent project of the instance.
 We are working to resolve this.

diff --git a/doc/explanation/storage.md b/doc/explanation/storage.md
@@ -23,6 +23,7 @@ The following storage drivers are supported:
 - [Ceph RBD - `ceph`](storage-ceph)
 - [CephFS - `cephfs`](storage-cephfs)
 - [Ceph Object - `cephobject`](storage-cephobject)
+- [Dell PowerFlex - `powerflex`](storage-powerflex)
 
 See the following how-to guides for additional information:
 
@@ -35,12 +36,12 @@ See the following how-to guides for additional information:
 Where the LXD data is stored depends on the configuration and the selected storage driver.
 Depending on the storage driver that is used, LXD can either share the file system with its host or keep its data separate.
 
-Storage location         | Directory | Btrfs    | LVM      | ZFS      | Ceph (all) |
-:---                     | :-:       | :-:      | :-:      | :-:      | :-:        |
-Shared with the host     | &#x2713;  | &#x2713; | -        | &#x2713; | -          |
-Dedicated disk/partition | -         | &#x2713; | &#x2713; | &#x2713; | -          |
-Loop disk                | -         | &#x2713; | &#x2713; | &#x2713; | -          |
-Remote storage           | -         | -        | -        | -        | &#x2713;   |
+Storage location         | Directory | Btrfs    | LVM      | ZFS      | Ceph (all) | Dell PowerFlex |
+:---                     | :-:       | :-:      | :-:      | :-:      | :-:        | :-:            |
+Shared with the host     | &#x2713;  | &#x2713; | -        | &#x2713; | -          | -              |
+Dedicated disk/partition | -         | &#x2713; | &#x2713; | &#x2713; | -          | -              |
+Loop disk                | -         | &#x2713; | &#x2713; | &#x2713; | -          | -              |
+Remote storage           | -         | -        | -        | -        | &#x2713;   | &#x2713;       |
 
 #### Shared with the host
 
@@ -70,6 +71,7 @@ You can increase their size though; see {ref}`storage-resize-pool`.
 #### Remote storage
 
 The `ceph`, `cephfs` and `cephobject` drivers store the data in a completely independent Ceph storage cluster that must be set up separately.
+The same applies to the `powerflex` driver.
 
 (storage-default-pool)=
 ### Default storage pool

diff --git a/doc/howto/import_machines_to_instances.md b/doc/howto/import_machines_to_instances.md
@@ -27,23 +27,33 @@ The tool can create both containers and virtual machines:
 
 * When creating a container, you must provide a disk or partition that contains the root file system for the container.
   For example, this could be the `/` root disk of the machine or container where you are running the tool.
-* When creating a virtual machine, you must provide a bootable disk, partition or image.
+* When creating a virtual machine, you must provide a bootable disk, partition, or an image in raw, QCOW, QCOW2, VDI, VHDX, or VMDK format.
   This means that just providing a file system is not sufficient, and you cannot create a virtual machine from a container that you are running.
   It is also not possible to create a virtual machine from the physical machine that you are using to do the migration, because the migration tool would be using the disk that it is copying.
   Instead, you could provide a bootable image, or a bootable partition or disk that is currently not in use.
 
+The tool can also inject the required VIRTIO drivers into the image:
+
+* To convert the image into raw format and inject the VIRTIO drivers during the conversion, use the following command:
+
+      lxd-migrate --conversion=format,virtio
+
+  ```{note}
+  The conversion option `virtio` requires `virt-v2v-in-place` to be installed on the host where the LXD server runs.
+  ```
+
+* For converting Windows images from a foreign hypervisor (not from QEMU/KVM with Q35/`virtio-scsi`), you must install additional drivers on the host:
+   * Install the `virtio-win` package or download the [`virtio-win.iso`](https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso) file and place it in the `/usr/share/virtio-win` directory.
+   * Download [`rhsrvany.exe` and `pnp_wait.exe`](https://github.com/rwmjones/rhsrvany?tab=readme-ov-file#binary-releases), and place them in the `/usr/share/virt-tools/` directory.
+
    ````{tip}
-   If you want to convert a Windows VM from a foreign hypervisor (not from QEMU/KVM with Q35/`virtio-scsi`),
-   you must install the `virtio-win` drivers to your Windows. Otherwise, your VM won't boot.
-   <details>
-   <summary>Expand to see how to integrate the required drivers to your Windows VM</summary>
-   Install the required tools on the host:
+   If you want to convert a Windows VM from a foreign hypervisor manually,
+   you must install both the required Windows drivers (as described above) and `virt-v2v` (>= 2.3.4).
 
-   1. Install `virt-v2v` version >= 2.3.4 (this is the minimal version that supports the `--block-driver` option).
-   1. Install the `virtio-win` package, or download the [`virtio-win.iso`](https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso) image and put it into the `/usr/share/virtio-win` folder.
-   1. You might also need to install [`rhsrvany`](https://github.com/rwmjones/rhsrvany).
+   <details>
+   <summary>Expand to see how to convert your Windows VM using <code>virt-v2v</code></summary>
 
-   Now you can use `virt-v2v` to convert images from a foreign hypervisor to `raw` images for LXD and include the required drivers:
+   Use `virt-v2v` to convert Windows image into `raw` format and include the required drivers.
 
    ```
    # Example 1. Convert a vmdk disk image to a raw image suitable for lxd-migrate
@@ -53,6 +63,7 @@ The tool can create both containers and virtual machines:
    ```
 
    You can find the resulting image in the `os` directory and use it with `lxd-migrate` on the next steps.
+   In addition, when migrating already converted images, `lxd-migrate` conversion options are not necessary.
    </details>
    ````
 

diff --git a/doc/howto/instances_create.md b/doc/howto/instances_create.md
@@ -74,6 +74,19 @@ To start an instance, send a PUT request to change the instance state:
 
 See {ref}`instances-manage-start` for more information.
 
+If you would like to start the instance upon creation, set the `start` property to true. The following example will create the container, then start it:
+
+    lxc query --request POST /1.0/instances --data '{
+      "name": "<instance_name>",
+      "source": {
+        "alias": "<image_alias>",
+        "protocol": "simplestreams",
+        "server": "<server_URL>",
+        "type": "image"
+      },
+      "start": true
+    }'
+
 ````
 
 ````{group-tab} UI

diff --git a/doc/howto/network_acls.md b/doc/howto/network_acls.md
@@ -42,8 +42,8 @@ Valid network ACL names must adhere to the following rules:
 
 ACLs have the following properties:
 
-% Include content from [../config_options.txt](../config_options.txt)
-```{include} ../config_options.txt
+% Include content from [../metadata.txt](../metadata.txt)
+```{include} ../metadata.txt
     :start-after: <!-- config group network-acl-acl-properties start -->
     :end-before: <!-- config group network-acl-acl-properties end -->
 ```
@@ -92,8 +92,8 @@ If one of the rules in the ACLs matches, the action for that rule is taken and n
 
 ACL rules have the following properties:
 
-% Include content from [../config_options.txt](../config_options.txt)
-```{include} ../config_options.txt
+% Include content from [../metadata.txt](../metadata.txt)
+```{include} ../metadata.txt
     :start-after: <!-- config group network-acl-rule-properties start -->
     :end-before: <!-- config group network-acl-rule-properties end -->
 ```

diff --git a/doc/howto/network_forwards.md b/doc/howto/network_forwards.md
@@ -50,8 +50,8 @@ Note that this target address must be within the same subnet as the network that
 
 Network forwards have the following properties:
 
-% Include content from [../config_options.txt](../config_options.txt)
-```{include} ../config_options.txt
+% Include content from [../metadata.txt](../metadata.txt)
+```{include} ../metadata.txt
     :start-after: <!-- config group network-forward-forward-properties start -->
     :end-before: <!-- config group network-forward-forward-properties end -->
 ```
@@ -94,8 +94,8 @@ If you want to forward the traffic to different ports, you have two options:
 
 Network forward ports have the following properties:
 
-% Include content from [../config_options.txt](../config_options.txt)
-```{include} ../config_options.txt
+% Include content from [../metadata.txt](../metadata.txt)
+```{include} ../metadata.txt
     :start-after: <!-- config group network-forward-port-properties start -->
     :end-before: <!-- config group network-forward-port-properties end -->
 ```

diff --git a/doc/howto/network_load_balancers.md b/doc/howto/network_load_balancers.md
@@ -35,8 +35,8 @@ If a listen address is not given, the `--allocate` flag must be provided.
 
 Network load balancers have the following properties:
 
-% Include content from [../config_options.txt](../config_options.txt)
-```{include} ../config_options.txt
+% Include content from [../metadata.txt](../metadata.txt)
+```{include} ../metadata.txt
     :start-after: <!-- config group network-load-balancer-load-balancer-properties start -->
     :end-before: <!-- config group network-load-balancer-load-balancer-properties end -->
 ```
@@ -74,8 +74,8 @@ If you want to forward the traffic to different ports, you have two options:
 
 Network load balancer backends have the following properties:
 
-% Include content from [../config_options.txt](../config_options.txt)
-```{include} ../config_options.txt
+% Include content from [../metadata.txt](../metadata.txt)
+```{include} ../metadata.txt
     :start-after: <!-- config group network-load-balancer-load-balancer-backend-properties start -->
     :end-before: <!-- config group network-load-balancer-load-balancer-backend-properties end -->
 ```
@@ -98,8 +98,8 @@ The backend(s) specified must have target port(s) settings compatible with the p
 
 Network load balancer ports have the following properties:
 
-% Include content from [../config_options.txt](../config_options.txt)
-```{include} ../config_options.txt
+% Include content from [../metadata.txt](../metadata.txt)
+```{include} ../metadata.txt
     :start-after: <!-- config group network-load-balancer-load-balancer-port-properties start -->
     :end-before: <!-- config group network-load-balancer-load-balancer-port-properties end -->
 ```

diff --git a/doc/howto/network_ovn_peers.md b/doc/howto/network_ovn_peers.md
@@ -46,8 +46,8 @@ This behavior prevents users in a different project from discovering whether a p
 
 Peer routing relationships have the following properties:
 
-% Include content from [../config_options.txt](../config_options.txt)
-```{include} ../config_options.txt
+% Include content from [../metadata.txt](../metadata.txt)
+```{include} ../metadata.txt
     :start-after: <!-- config group network-peering-peering-properties start -->
     :end-before: <!-- config group network-peering-peering-properties end -->
 ```

diff --git a/doc/howto/network_zones.md b/doc/howto/network_zones.md
@@ -155,8 +155,8 @@ lxc network zone edit <network_zone>
 
 The following configuration options are available for network zones:
 
-% Include content from [../config_options.txt](../config_options.txt)
-```{include} ../config_options.txt
+% Include content from [../metadata.txt](../metadata.txt)
+```{include} ../metadata.txt
     :start-after: <!-- config group network-zone-config-options start -->
     :end-before: <!-- config group network-zone-config-options end -->
 ```
@@ -205,8 +205,8 @@ This command creates an empty record without entries and adds it to a network zo
 
 Records have the following properties:
 
-% Include content from [../config_options.txt](../config_options.txt)
-```{include} ../config_options.txt
+% Include content from [../metadata.txt](../metadata.txt)
+```{include} ../metadata.txt
     :start-after: <!-- config group network-zone-record-properties start -->
     :end-before: <!-- config group network-zone-record-properties end -->
 ```
-Original file line number
+Diff line change
@@ Expand Up / @@ -39,6 +39,7 @@ OEM @@
     OLM
     Permalink
     pre
+    QCow
     Quickstart
     ReadMe
     reST
@@ Expand All / @@ -50,5 +51,9 @@ subtree @@
     Ubuntu
     UI
     UUID
+    VDI
+    VHDX
+    VIRTIO
     VM
+    VMDK
     YAML