Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Longhorn v24.09 #37

Merged
merged 24 commits into from
Nov 15, 2024
Merged

Conversation

DamiaSan
Copy link

@DamiaSan DamiaSan commented Nov 12, 2024

Which issue(s) this PR fixes:

Issue longhorn/longhorn#9700

What this PR does / why we need it:

New branch based on SPDK 24.09 release

Special notes for your reviewer:

Additional documentation or context

DamiaSan and others added 24 commits November 5, 2024 17:01
The minimun number of raid1 base bdevs is set to 1, so a raid1 bdev
can be created with only one base bdev.

Signed-off-by: Damiano Cipriani <[email protected]>
The returned value write_string_or_name() is not really checked, so one double quote char error
is not identified and the broken json string is directly sent to the client.

To address the issue, the workaround is always appending one double quote char in the end of string.

TODO:
1. Find the root cause
2. Check the returned value of write_string_or_name()

Longhorn 6190

Signed-off-by: Derek Su <[email protected]>
Refactored the code creating raid_bdev_module_get_min_operational,
distinguished intentional removal from failure in superblock base
bdev state.

Signed-off-by: Damiano Cipriani <[email protected]>
Implemented the functionality to add a new base bdev to an existing
raid bdev, growing the size of the raid if needed.
Actually only raid1 supports growing with a new base bdev.

Signed-off-by: Damiano Cipriani <[email protected]>
Implemented the RPC to add a new base bdev to a raid bdev,
growing the size of the raid if there isn't an empty base bdev slot.

Signed-off-by: Damiano Cipriani <[email protected]>
Get a fragmap for a specific segment of a logical volume using the provided offset and size.
A fragmap is a bitmap that records the allocation status of clusters. A value of "1" indicates
that a cluster is allocated, whereas "0" signifies that a cluster is unallocated.

Longhorn 6138

Signed-off-by: Derek Su <[email protected]>
Longhorn 6138

Signed-off-by: Derek Su <[email protected]>
Longhorn 6604

Signed-off-by: Derek Su <[email protected]>
Longhorn 6604

Signed-off-by: Derek Su <[email protected]>
Longhorn 6604

Signed-off-by: Derek Su <[email protected]>
Flush I/O has been implementd in raid1 simply routing the flush
command to all base bdevs composing the raid bdev.

Signed-off-by: Damiano Cipriani <[email protected]>
This function creates a snapshot taking as additional parameter
a list of xattrs for the newly created snapshot. Once the
snapshot is created, its metadata become read only, so we can
set xattrs only during the creation.

Signed-off-by: Damiano Cipriani <[email protected]>
New params have been added to vbdev_lvol_create_snapshot to
set xattrs in the newly created snapshot.

Signed-off-by: Damiano Cipriani <[email protected]>
A new option has been added to the bdev_lvol_snapshot RPC to set
xattrs in the newly created snapshot.

Signed-off-by: Damiano Cipriani <[email protected]>
…a single block device

- disk-status prints the disk status by json-formatted string.
    ```
    0000:00:03.0 (1af4 1001): Active devices: holder@vda3:dm-0,mount@vda:vda2,mount@vda:ubuntu--vg-ubuntu--lv, so not binding PCI dev
    {"bdf":"0000:00:04.0","type":"virtio","driver":"virtio-pci","vendor":"1af4","numa":"unknown","device":"-","block_devices":"vdb"}
    ```

- disk-driver prints driver name associated with a given PCI device's BDF address.
    Example:
    ```
    bash ./setup.sh get-device-driver 0000:00:04.0
    ```

- bind binds a PCI device to a specific driver.
    Example for binding a device:
    ```
    export PCI_ALLOWED=0000:00:04.0; export DRIVER_OVERRIDE=uio_pci_generic; bash ./setup.sh bind
    ```

- unbind unbinds a PCI device from its current driver and rebind it to the original driver
    Example for unbinding a device:
    ```
    bash ./setup.sh unbind 0000:00:04.0
    ```

Longhorn 7672

Signed-off-by: Derek Su <[email protected]>
UNMAP I/O has been implementd in raid1 simply routing the UNMAP
command to all base bdevs composing the raid bdev.

Longhorn 7925

Signed-off-by: Derek Su <[email protected]>
drivers_d is already replaced with pci_bus_driver in spdk@4f8177b

Signed-off-by: Derek Su <[email protected]>
A delta bitmap for fast rebuild is created and updated for a defined
amout of time when the bdev of a base bdev is deleted or when it gets
write I/O error.

Longhorn 5573

Signed-off-by: Damiano Cipriani <[email protected]>
Older environment may not support RWF_NOWAIT, do not apply the flag if it is not supported.

Change-Id: I40e83ef52aa3842c63bc41f48c4c57eaecd48ac1
Signed-off-by: Jinlong Chen <[email protected]>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25372
Reviewed-by: Jim Harris <[email protected]>
Tested-by: SPDK CI Jenkins <[email protected]>
Reviewed-by: Konrad Sztyber <[email protected]>
Reviewed-by: GangCao <[email protected]>
Community-CI: Community CI Samsung <[email protected]>
Community-CI: Broadcom CI <[email protected]>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <[email protected]>
With Linux loop device, occasionally, we continuosly get an EAGAIN
error, so entering in an infinite loop.

Longhorn 9700

Signed-off-by: Damiano Cipriani <[email protected]>
@innobead
Copy link
Member

Which issue(s) this PR fixes:

Issue longhorn/longhorn#9700

@DamiaSan please also remember to add a reference link to the source issue instead of the issue number only, so we can clearly understand what ongoing PRs are.

@derekbit
Copy link
Member

derekbit commented Nov 14, 2024

@DamiaSan Is the lvstore creation stuck issue resolved?

@DamiaSan
Copy link
Author

@DamiaSan Is the lvstore creation stuck issue resolved?

Yes, I have simply disabled the commit which cause the issue.

@derekbit
Copy link
Member

@DamiaSan Is the lvstore creation stuck issue resolved?

Yes, I have simply disabled the commit which cause the issue.

I see, but what's the original purpose of adding the flag RWF_NOWAIT`?
Any side effect if we disable it?

@DamiaSan
Copy link
Author

It is a new feature introduced with this commit. After this, with another commit, they disabled the functionality where it is not supported. I have simply disdabled it until we are not able to understand the problem and fix it, there shouldn't be any side effects because until Septermber the functionality was not present in code base and it is disabled in environment where it is not supported.

@derekbit derekbit merged commit 10463b5 into longhorn:longhorn-v24.09 Nov 15, 2024
1 check passed
@derekbit
Copy link
Member

derekbit commented Nov 15, 2024

@DamiaSan I have marked the longhorn-v24.09 as the default branch. Then, you can create PRs for updating the dependency in go-spdk-helper, longhorn-spdk-helper and longhorn-instance-manager.

@DamiaSan
Copy link
Author

@DamiaSan I have marked the longhorn-v24.09 as the default branch. Then, you can create PRs for updating the dependency in go-spdk-helper, longhorn-spdk-helper and longhorn-instance-manager.

For go-spdk-helper and longhorn-spdk-engine, the updates are in the PRs for the delta bitmap: longhorn/go-spdk-helper#163 and longhorn/longhorn-spdk-engine#238. Now that the new branch has been merged I will convert these drafts into real PRS.
Ok for longhorn-instance-manager.

@derekbit
Copy link
Member

derekbit commented Nov 15, 2024

@DamiaSan I have marked the longhorn-v24.09 as the default branch. Then, you can create PRs for updating the dependency in go-spdk-helper, longhorn-spdk-helper and longhorn-instance-manager.

For go-spdk-helper and longhorn-spdk-engine, the updates are in the PRs for the delta bitmap: longhorn/go-spdk-helper#163 and longhorn/longhorn-spdk-engine#238. Now that the new branch has been merged I will convert these drafts into real PRS. Ok for longhorn-instance-manager.

Could you separate the PRs? Create separate PRs for the spdk dependency update.

@DamiaSan
Copy link
Author

Could you separate the PRs? Create separate PRs for the spdk dependency update

Sure that I can, but what is the benefit? In each PR there are separate commits, one for the dependency update and one for the delta bitmap.

@derekbit
Copy link
Member

derekbit commented Nov 15, 2024

Could you separate the PRs? Create separate PRs for the spdk dependency update

Sure that I can, but what is the benefit? In each PR there are separate commits, one for the dependency update and one for the delta bitmap.

It is helpful when we need to check the context of the PR and its ticket. The two commits serve for different and individual purposes, so I suggest separating them.

The spdk update PR should belong to longhorn/longhorn#9700.

@DamiaSan
Copy link
Author

Ok, I will separate them.
However I suggest to not keep under review a PR for about 4 months, as for this one, because this is the root cause of such a complication.

@derekbit
Copy link
Member

I suggest to not keep under review a PR for about 4 months, as for this one, because this is the root cause of such a complication.

Yes, everyone's bandwidth has been very tight in recent months. Please ping other members if the PR is urgent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants