Please clarify install instructions and requirements (docs say TBD a lot)

[bix6]

SUMMARY

Please clarify install instructions and requirements. I am struggling to setup this module and cannot find clear documentation anywhere. I am attempting to hack multiple guides together and still not having success. I have tried both python2 and python3 now.

Here is the current flow I am attempting. It's still not working but I don't know where to go from here.

sudo apt update
sudo apt -y install python3-pip
python3 -m pip install -U pip
python3 -m pip install --user ansible
// I think pkg-config may come with libvirt-dev as well
sudo apt install pkg-config
sudo apt install libvirt-dev
python3 -m pip install libvirt-python

// Update path
nano ~/.bashrc
// Add this
export PATH=$PATH:$HOME/.local/bin
// Refresh bash
source ~/.bashrc

This is the error I have now in my playbook (I have read many pages discussing this error):

{"changed": false, "msg": "The `libvirt` module is not importable. Check the requirements."}

ISSUE TYPE

• Documentation Report

COMPONENT NAME

community.libvirt.virt

ANSIBLE VERSION

[DEPRECATION WARNING]: Ansible will require Python 3.8 or newer on the controller starting with Ansible 2.12. Current 
version: 3.6.9 (default, Jan 26 2021, 15:33:00) [GCC 8.4.0]. This feature will be removed from ansible-core in version 
2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
ansible [core 2.11.1] 
  config file = None
  configured module search path = ['/home/ubuntu/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/ubuntu/.local/lib/python3.6/site-packages/ansible
  ansible collection location = /home/ubuntu/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/ubuntu/.local/bin/ansible
  python version = 3.6.9 (default, Jan 26 2021, 15:33:00) [GCC 8.4.0]
  jinja version = 2.10
  libyaml = True

[ceesios]

Hi,
I ran into the same problem today. I overlooked the fact that ansible is running on the remote host, and that is where the requirements should be installed. So either add a task to install the requirements remotely or use delegate_to to run the task locally.

[dtantsur]

On top of that, could you turn "Check the requirements" into something more verbose? Maybe the actual ImportError?

We have a very recent breakage on CentOS 8 (Stream), and I don't know what causes it. The requirements used to be fine just a couple of weeks ago.

[Andersson007]

@dtantsur hi, thanks for reporting this!
You installed libvirt-python via pip3. Could you please try run your playbook with the -e "ansible_python_interpreter=/path/to/your/python3/interpreter" argument? (Python 3 support)
Looks like the module uses python2 and doesn't see python3 libs.

I'm looking forward to your feedback

[dtantsur]

@Andersson007 that's what we're doing. We're well past Python 3 issues at this point (we switched more than a year ago).

This is a workaround we have to provide: https://review.opendev.org/c/openstack/bifrost/+/800214/3/playbooks/roles/bifrost-create-vm-nodes/tasks/prepare_libvirt.yml. It essentially prevents pip from using libvirt wheels and forces a source installation. I must admit, I have no clue how and why it works, especially since this issue is not trivial to reproduce (we hit it in the CI workers, but not locally).

[Andersson007]

@dtantsur thank you for the feedback and the workaround!
I pinged the folks in #77 (comment). Maybe they have some ideas and will respond. We should definitely improve this part of the doc.

[odyssey4me]

I would caution everyone that mixing your system packages and packages from pypi is not a good idea. Things from pypi should be installed in a user's home directory (not root) using --user or should go into a venv.

libvirt-python is a bit of a pain because it has host OS binary bindings, so usually it is best to install it via the distribution packaging (a package called python-libvirt or something like that). This package must be on the target host where the virtual machines will be run or where the inventory source is.

If you want to use it in a venv, then there are two ways I've done this before:

1. Install the distribution package for libvirt/python-libvirt, then create the venv with --system-site-packages so that the system libraries are put into the venv.
2. Use something like https://opendev.org/openstack/ansible-role-python_venv_build/src/commit/82dcd020fa950d1df3a2a71e263cb0cce3386874/tasks/python_venv_install_symlink.yml#L24-L44 to create your own symlinks from the venv to the host libraries.

Be warned that option 1 above is simpler, but will bring in more libraries. Option 2 is more complex, but also more surgical.

[Andersson007]

Could anyone try the way suggested by @odyssey4me and give feedback?
If we had a step-by-step guide, we could create INSTALLATION file and link to it from README (or put the guide directly to README if it's not long).

[dtantsur]

I would caution everyone that mixing your system packages and packages from pypi is not a good idea. Things from pypi should be installed in a user's home directory (not root) using --user or should go into a venv.

Yep, except that python-apt/python-dnf are not distributed via pypi, so to use ansible with venv you need to use --system-site-packages. Which could contribute to our issues :(

[Andersson007]

(Off topic: there's the Quick-start guide there which can help people who want to clarify the installation process but have no experience in contributing to Ansible and how to submit a pull request. All related to testing in the guide is irrelevant here)