Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem with firmware upload after changing from aoscx role to collection #82

Open
jon-larsen opened this issue Sep 20, 2023 · 19 comments
Open

Problem with firmware upload after changing from aoscx role to collection #82

jon-larsen opened this issue Sep 20, 2023 · 19 comments
Assignees
@tchiapuziowong
Labels
bug Something isn't working

Comments

@jon-larsen
Copy link

A couple of months ago I changed all ansible-playbooks from using the aoscx role to using the aoscx collection. All of them have been working fine, except for the playbook for uploading firmware. I didn't find this out until now, so I am not completely sure if there could be any other reason than changing to collection.

Switch: 6200F
Firmware: ML.10.10.1070

This is the error I receive:
GENERIC OPERATION ERROR: Command not allowed.\\n: Code: 403
It takes about a minute before the error appears, so it seems Ansible is working with something before it fails.

https-server rest access-mode read-write is configured on the switch (not visible in the config since it is default).

I have a group_vars for the switches which contains this:

ansible_network_os: arubanetworks.aoscx.aoscx
ansible_connection: arubanetworks.aoscx.aoscx
ansible_aoscx_validate_certs: False
ansible_aoscx_use_proxy: False
ansible_httpapi_use_ssl: True
ansible_httpapi_validate_certs: False

The user running Ansible have access to the firmware file. I have tested copying the path from the ansible output and using ls and cat on the filepath with the user.

playbook 
---
- hosts: osl-sw-tele-03
collections:
  - arubanetworks.aoscx
gather_facts: False
any_errors_fatal: true
tasks:

# Check current firmware version
  - name: Get current firmware version
    arubanetworks.aoscx.aoscx_facts:
      gather_subset: ['software_version','software_images']
    register: fact_current_version

  - name: Set current_version variable
    set_fact:
      current_version: "{{ fact_current_version['ansible_facts']['ansible_net_software_version'] }}"

  - name: Set image_version variable
    set_fact:
      image_version: "{{ fact_current_version['ansible_facts']['ansible_net_software_images'][boot_partition + '_image_version'] }}"

# Upload firmware (unless firmware already exists)
  - name: Upload firmware through local file
    arubanetworks.aoscx.aoscx_upload_firmware:
      partition_name: '{{ boot_partition }}'
      firmware_file_path: '/data/github-runner/firmware/{{ firmware_filename }}'
    when: image_version is not regex('..\.' + firmware)
playbook output 
playbook -f 5 firmware-upload.yml -i inventory -vvv
***-playbook [core 2.12.10]
config file = /etc/***/***.cfg
configured module search path = ['/home/github/.***/plugins/modules', '/usr/share/***/plugins/modules']
*** python module location = /usr/lib/python3/dist-packages/***
*** collection location = /home/github/.***/collections:/usr/share/***/collections
executable location = /usr/bin/***-playbook
python version = 3.8.10 (default, May 26 2023, 14:05:08) [GCC 9.4.0]
jinja version = 2.10.1
libyaml = True
Using /etc/***/***.cfg as config file
host_list declined parsing /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/inventory as it did not pass its verify_file() method
script declined parsing /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/inventory as it did not pass its verify_file() method
auto declined parsing /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/inventory as it did not pass its verify_file() method
Parsed /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/inventory inventory source with yaml plugin
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: firmware-upload.yml **************************************************
1 plays in firmware-upload.yml

PLAY [osl-sw-tele-03] **********************************************************
META: ran handlers

TASK [Get current firmware version] ********************************************
task path: /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/firmware-upload.yml:10
<osl-sw-tele-03> ESTABLISH LOCAL CONNECTION FOR USER: github
<osl-sw-tele-03> EXEC /bin/sh -c '( umask [7](https://github.com/arkivverket/drift-nettverk-switch-firmware/actions/runs/6247900006/job/16961400965#step:3:8)7 && mkdir -p "` echo /home/github/.***/tmp/***-local-442964li06uwjh `"&& mkdir "` echo /home/github/.***/tmp/***-local-442964li06uwjh/***-tmp-1695209262.0190933-442969-11[8](https://github.com/arkivverket/drift-nettverk-switch-firmware/actions/runs/6247900006/job/16961400965#step:3:9)[9](https://github.com/arkivverket/drift-nettverk-switch-firmware/actions/runs/6247900006/job/16961400965#step:3:10)77890285825 `" && echo ***-tmp-1695209262.0190933-442969-118977890285825="` echo /home/github/.***/tmp/***-local-442964li06uwjh/***-tmp-1695209262.0190933-442969-118977890285825 `" ) && sleep 0'
Using module file /home/github/.***/collections/***_collections/arubanetworks/aoscx/plugins/modules/aoscx_facts.py
<osl-sw-tele-03> PUT /home/github/.***/tmp/***-local-442964li06uwjh/tmpiey3r_5y TO /home/github/.***/tmp/***-local-442964li06uwjh/***-tmp-1695209262.0190933-442969-118977890285825/AnsiballZ_aoscx_facts.py
<osl-sw-tele-03> EXEC /bin/sh -c 'chmod u+x /home/github/.***/tmp/***-local-442964li06uwjh/***-tmp-1695209262.0190933-442969-118977890285825/ /home/github/.***/tmp/***-local-442964li06uwjh/***-tmp-1695209262.0190933-442969-118977890285825/AnsiballZ_aoscx_facts.py && sleep 0'
<osl-sw-tele-03> EXEC /bin/sh -c '/usr/bin/python3 /home/github/.***/tmp/***-local-442964li06uwjh/***-tmp-1695209262.0190933-442969-118977890285825/AnsiballZ_aoscx_facts.py && sleep 0'
<osl-sw-tele-03> EXEC /bin/sh -c 'rm -f -r /home/github/.***/tmp/***-local-442964li06uwjh/***-tmp-1695209262.0190933-442969-118977890285825/ > /dev/null 2>&1 && sleep 0'
ok: [osl-sw-tele-03] => {
  "***_facts": {
      "***_net_gather_network_resources": null,
      "***_net_gather_subset": [
          "software_version",
          "software_images",
          "management_interface",
          "software_version"
      ],
      "***_net_mgmt_intf_status": {
          "domain_name": "",
          "hostname": "osl-sw-tele-03",
          "link_state": "down",
          "mac_address": "b8:d4:e7:b0:2e:81"
      },
      "***_net_software_images": {
          "boot_profile_timeout": 5,
          "default_image": "secondary",
          "primary_image_date": "2023-03-27 14:44:26 UTC",
          "primary_image_sha_256": "027ae3cdc3dad79d96e6bfd44cef8330e6f69e06429d7b3ca8592605a3e4b182",
          "primary_image_size": "456 MB",
          "primary_image_version": "ML.[10](https://github.com/arkivverket/drift-nettverk-switch-firmware/actions/runs/6247900006/job/16961400965#step:3:11).10.1050",
          "secondary_image_date": "2023-06-20 14:48:32 UTC",
          "secondary_image_sha_256": "2f4fab7a24d4ebdf71dd5b9fc0d07e2385a6b8a4e3fe28a902a6fa582f96107e",
          "secondary_image_size": "456 MB",
          "secondary_image_version": "ML.10.10.1070"
      },
      "***_net_software_version": "ML.10.10.1070",
      "***_network_resources": {}
  },
  "changed": false,
  "invocation": {
      "module_args": {
          "gather_network_resources": null,
          "gather_subset": [
              "software_version",
              "software_images",
              "management_interface",
              "software_version"
          ]
      }
  }
}

TASK [Set current_version variable] ********************************************
task path: /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/firmware-upload.yml:15
ok: [osl-sw-tele-03] => {
  "***_facts": {
      "current_version": "ML.10.10.1070"
  },
  "changed": false
}

TASK [Set image_version variable] **********************************************
task path: /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/firmware-upload.yml:19
ok: [osl-sw-tele-03] => {
  "***_facts": {
      "image_version": "ML.10.10.1050"
  },
  "changed": false
}

TASK [Upload firmware through local file] **************************************
task path: /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/firmware-upload.yml:24
<osl-sw-tele-03> ESTABLISH LOCAL CONNECTION FOR USER: github
<osl-sw-tele-03> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/github/.***/tmp/***-local-442964li06uwjh `"&& mkdir "` echo /home/github/.***/tmp/***-local-442964li06uwjh/***-tmp-1695209264.3027976-4430[11](https://github.com/arkivverket/drift-nettverk-switch-firmware/actions/runs/6247900006/job/16961400965#step:3:12)-2514[12](https://github.com/arkivverket/drift-nettverk-switch-firmware/actions/runs/6247900006/job/16961400965#step:3:13)824569301 `" && echo ***-tmp-1695209264.3027976-443011-25[14](https://github.com/arkivverket/drift-nettverk-switch-firmware/actions/runs/6247900006/job/16961400965#step:3:15)12824569301="` echo /home/github/.***/tmp/***-local-442964li06uwjh/***-tmp-[16](https://github.com/arkivverket/drift-nettverk-switch-firmware/actions/runs/6247900006/job/16961400965#step:3:17)95209264.3027976-443011-251412824569301 `" ) && sleep 0'
Using module file /home/github/.***/collections/***_collections/arubanetworks/aoscx/plugins/modules/aoscx_upload_firmware.py
<osl-sw-tele-03> PUT /home/github/.***/tmp/***-local-442964li06uwjh/tmpbw_mbkho TO /home/github/.***/tmp/***-local-442964li06uwjh/***-tmp-1695209264.3027976-443011-251412824569301/AnsiballZ_aoscx_upload_firmware.py
<osl-sw-tele-03> EXEC /bin/sh -c 'chmod u+x /home/github/.***/tmp/***-local-442964li06uwjh/***-tmp-1695209264.3027976-443011-251412824569301/ /home/github/.***/tmp/***-local-442964li06uwjh/***-tmp-1695209264.3027976-443011-251412824569301/AnsiballZ_aoscx_upload_firmware.py && sleep 0'
<osl-sw-tele-03> EXEC /bin/sh -c '/usr/bin/python3 /home/github/.***/tmp/***-local-442964li06uwjh/***-tmp-1695209264.3027976-443011-251412824569301/AnsiballZ_aoscx_upload_firmware.py && sleep 0'
<osl-sw-tele-03> EXEC /bin/sh -c 'rm -f -r /home/github/.***/tmp/***-local-442964li06uwjh/***-tmp-1695209264.3027976-443011-251412824569301/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
File "/tmp/***_arubanetworks.aoscx.aoscx_upload_firmware_payload_jg9sfgpx/***_arubanetworks.aoscx.aoscx_upload_firmware_payload.zip/***_collections/arubanetworks/aoscx/plugins/modules/aoscx_upload_firmware.py", line 167, in main
File "/home/github/.local/lib/python3.8/site-packages/pyaoscx/device.py", line 565, in upload_firmware
  result = self.upload_firmware_local(
File "/home/github/.local/lib/python3.8/site-packages/pyaoscx/device.py", line 5[18](https://github.com/arkivverket/drift-nettverk-switch-firmware/actions/runs/6247900006/job/16961400965#step:3:19), in upload_firmware_local
  success = utils.file_upload(
File "/home/github/.local/lib/python3.8/site-packages/pyaoscx/utils/util.py", line 309, in file_upload
  raise GenericOperationError(
fatal: [osl-sw-tele-03]: FAILED! => {
  "changed": false,
  "invocation": {
      "module_args": {
          "firmware_file_path": "/data/github-runner/firmware/ArubaOS-CX_6[20](https://github.com/arkivverket/drift-nettverk-switch-firmware/actions/runs/6247900006/job/16961400965#step:3:21)0_10_10_1080.swi",
          "partition_name": "primary",
          "remote_firmware_file_path": null,
          "vrf": null,
          "wait_firmware_upload": false
      }
  },
  "msg": "'GENERIC OPERATION ERROR: Command not allowed.\\n: Code: [40](https://github.com/arkivverket/drift-nettverk-switch-firmware/actions/runs/6247900006/job/16961400965#step:3:41)3'"
}

NO MORE HOSTS LEFT *************************************************************

PLAY RECAP *********************************************************************
osl-sw-tele-03             : ok=3    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0
collection versions 
$ ansible-galaxy collection list

# /home/github/.ansible/collections/ansible_collections
Collection               Version
------------------------ -------
ansible.netcommon        5.2.0  
ansible.utils            2.11.0 
arubanetworks.aos_switch 1.7.0  
arubanetworks.aoscx      4.3.0  

# /usr/lib/python3/dist-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    2.3.0  
ansible.netcommon             2.6.1  
ansible.posix                 1.4.0  
ansible.utils                 2.6.1  
ansible.windows               1.10.0 
arista.eos                    3.1.0  
awx.awx                       19.4.0 
azure.azcollection            1.13.0 
check_point.mgmt              2.3.0  
chocolatey.chocolatey         1.2.0  
cisco.aci                     2.2.0  
cisco.asa                     2.1.0  
cisco.dnac                    6.5.0  
cisco.intersight              1.0.19 
cisco.ios                     2.8.1  
cisco.iosxr                   2.9.0  
cisco.ise                     1.2.1  
cisco.meraki                  2.8.0  
cisco.mso                     1.4.0  
cisco.nso                     1.0.3  
cisco.nxos                    2.9.1  
cisco.ucs                     1.8.0  
cloud.common                  2.1.2  
cloudscale_ch.cloud           2.2.2  
community.aws                 2.6.1  
community.azure               1.1.0  
community.ciscosmb            1.0.5  
community.crypto              2.3.4  
community.digitalocean        1.20.0 
community.dns                 2.2.0  
community.docker              2.6.0  
community.fortios             1.0.0  
community.general             4.8.3  
community.google              1.0.0  
community.grafana             1.5.0  
community.hashi_vault         2.5.0  
community.hrobot              1.4.0  
community.kubernetes          2.0.1  
community.kubevirt            1.0.0  
community.libvirt             1.1.0  
community.mongodb             1.4.1  
community.mysql               2.3.8  
community.network             3.3.0  
community.okd                 2.2.0  
community.postgresql          1.7.4  
community.proxysql            1.4.0  
community.rabbitmq            1.2.1  
community.routeros            2.1.0  
community.sap                 1.0.0  
community.sap_libs            1.1.0  
community.skydive             1.0.0  
community.sops                1.2.2  
community.vmware              1.18.2 
community.windows             1.10.0 
community.zabbix              1.7.0  
containers.podman             1.9.3  
cyberark.conjur               1.1.0  
cyberark.pas                  1.0.14 
dellemc.enterprise_sonic      1.1.1  
dellemc.openmanage            4.4.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.17.0 
fortinet.fortimanager         2.1.5  
fortinet.fortios              2.1.6  
frr.frr                       1.0.4  
gluster.gluster               1.0.2  
google.cloud                  1.0.2  
hetzner.hcloud                1.6.0  
hpe.nimble                    1.1.4  
ibm.qradar                    1.0.3  
infinidat.infinibox           1.3.3  
infoblox.nios_modules         1.2.2  
inspur.sm                     1.3.0  
junipernetworks.junos         2.10.0 
kubernetes.core               2.3.2  
mellanox.onyx                 1.0.0  
netapp.aws                    21.7.0 
netapp.azure                  21.10.0
netapp.cloudmanager           21.18.0
netapp.elementsw              21.7.0 
netapp.ontap                  21.20.0
netapp.storagegrid            21.10.0
netapp.um_info                21.8.0 
netapp_eseries.santricity     1.3.0  
netbox.netbox                 3.7.1  
ngine_io.cloudstack           2.2.4  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.2  
openstack.cloud               1.8.0  
openvswitch.openvswitch       2.1.0  
ovirt.ovirt                   1.6.6  
purestorage.flasharray        1.13.0 
purestorage.flashblade        1.9.0  
sensu.sensu_go                1.13.1 
servicenow.servicenow         1.0.6  
splunk.es                     1.0.2  
t_systems_mms.icinga_director 1.30.0 
theforeman.foreman            2.2.0  
vmware.vmware_rest            2.2.0  
vyos.vyos                     2.8.0  
wti.remote                    1.0.4
ansible version 
$ ansible --version
ansible [core 2.12.10]
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/github/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3/dist-packages/ansible
ansible collection location = /home/github/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/bin/ansible
python version = 3.8.10 (default, May 26 2023, 14:05:08) [GCC 9.4.0]
jinja version = 2.10.1
libyaml = True
pip versions 
$ pip list
Package                Version             
---------------------- --------------------
ansible                5.10.0              
ansible-core           2.12.10             
ansible-pylibssh       0.2.0               
attrs                  19.3.0              
Automat                0.8.0               
bcrypt                 3.2.0               
blinker                1.4                 
certifi                2019.11.28          
certsrv                2.1.1               
cffi                   1.14.6              
chardet                3.0.4               
Click                  7.0                 
cloud-init             23.2.2              
colorama               0.4.3               
command-not-found      0.3                 
configobj              5.0.6               
constantly             15.1.0              
cryptography           2.8                 
dbus-python            1.2.16              
distro                 1.4.0               
distro-info            0.23+ubuntu1.1      
entrypoints            0.3                 
httplib2               0.14.0              
hyperlink              19.0.0              
idna                   2.8                 
importlib-metadata     1.5.0               
incremental            16.10.1             
Jinja2                 2.10.1              
jmespath               0.9.4               
jsonpatch              1.22                
jsonpointer            2.0                 
jsonschema             3.2.0               
jxmlease               1.0.3               
keyring                18.0.1              
language-selector      0.1                 
launchpadlib           1.10.13             
lazr.restfulclient     0.14.2              
lazr.uri               1.0.3               
lxml                   4.6.3               
MarkupSafe             1.1.0               
more-itertools         4.2.0               
ncclient               0.6.12              
netaddr                0.8.0               
netifaces              0.10.4              
ntlm-auth              1.1.0               
oauthlib               3.1.0               
packaging              20.3                
paramiko               2.7.2               
pexpect                4.6.0               
pip                    20.0.2              
pyaoscx                2.5.0               
pyasn1                 0.4.2               
pyasn1-modules         0.2.1               
pycparser              2.20                
pycrypto               2.6.1               
pycurl                 7.45.2              
PyGObject              3.36.0              
PyHamcrest             1.9.0               
PyJWT                  1.7.1               
pykerberos             1.1.14              
pymacaroons            0.13.0              
PyNaCl                 1.3.0               
pyOpenSSL              19.0.0              
pyparsing              2.4.6               
pyrsistent             0.15.5              
pyserial               3.4                 
python-apt             2.0.1+ubuntu0.20.4.1
python-debian          0.1.36+ubuntu1.1    
pywinrm                0.3.0               
PyYAML                 5.3.1               
requests               2.22.0              
requests-kerberos      0.12.0              
requests-ntlm          1.1.0               
requests-toolbelt      1.0.0               
requests-unixsocket    0.2.0               
resolvelib             0.5.4               
SecretStorage          2.3.1               
service-identity       18.1.0              
setuptools             45.2.0              
simplejson             3.16.0              
six                    1.14.0              
sos                    4.5.6               
ssh-import-id          5.10                
systemd-python         234                 
Twisted                18.9.0              
ubuntu-advantage-tools 8001                
ufw                    0.36                
unattended-upgrades    0.1                 
urllib3                1.25.8              
wadllib                1.3.3               
wheel                  0.34.2              
xmltodict              0.12.0              
zipp                   1.0.0               
zope.interface         4.7.1
switch logs 
2023-09-20T14:05:29.239715+02:00 osl-sw-tele-03 hpe-restd[1072]: Event|4602|LOG_INFO|AMM|-|Authentication succeeded for user ansible in session xxxxxx
2023-09-20T14:05:29.239967+02:00 osl-sw-tele-03 hpe-restd[1072]: Event|4655|LOG_INFO|AMM|-|User ansible logged in from x.x.x.x through REST session
2023-09-20T14:05:31.404310+02:00 osl-sw-tele-03 hpe-restd[1072]: Event|4607|LOG_INFO|AMM|-|Authorization succeeded for user ansible, for resource FirmwareUpdate, with action POST
2023-09-20T14:06:01.460325+02:00 osl-sw-tele-03 hpe-restd[1072]: Event|4608|LOG_INFO|AMM|-|Authorization allowed for user ansible, for resource SessionMgmt, with action POST
2023-09-20T14:06:01.462332+02:00 osl-sw-tele-03 hpe-restd[1072]: Event|4657|LOG_INFO|AMM|-|User ansible logged out of REST session from x.x.x.x
@tchiapuziowong
Copy link
Member

Hi @jon-larsen ! Can you confirm if the IP address you're using is on the mgmt vrf or another? I feel this may be similar to this issue which we are investigating internally: #78

@jon-larsen
Copy link
Author

Hi @jon-larsen ! Can you confirm if the IP address you're using is on the mgmt vrf or another? I feel this may be similar to this issue which we are investigating internally: #78

The IP belongs to a VLAN interface which has no vrf configured.

@jon-larsen
Copy link
Author

Tested my playbook on an 8325 switch which is using the mgmt vrf, and it works on that one.

@tchiapuziowong
Copy link
Member

@jon-larsen thank you for that verification! I believe this is similar to #78 and we're currently investigating a fix which is anticipated in an upcoming minor release - I'll update this issue once we've implemented a fix. Thank you for your patience!

@tchiapuziowong tchiapuziowong self-assigned this Sep 25, 2023
@tchiapuziowong tchiapuziowong added the bug Something isn't working label Sep 25, 2023
@tchiapuziowong
Copy link
Member

Hello @jon-larsen ! We've just published a fix for this issue, please install the latest version of the collection and verify if you no longer see this issue:

ansible-galaxy collection install arubanetworks.aoscx -f

@jon-larsen
Copy link
Author

I got a new error message now:
Could not get PYAOSCX Session: 'LOGIN ERROR, Cookies were not set correctly. Login failed'

@tchiapuziowong
Copy link
Member

@jon-larsen which pyaoscx version is installed?

@jon-larsen
Copy link
Author

I had 2.5.0 at first, but I got the same error with 2.5.1

@tchiapuziowong
Copy link
Member

@jon-larsen after looking at your original issue the remote file path is this value:
"firmware_file_path": "/data/github-runner/firmware/ArubaOS-CX_6[20](https://github.com/arkivverket/drift-nettverk-switch-firmware/actions/runs/6247900006/job/16961400965#step:3:21)0_10_10_1080.swi"

is that correct? to have that github link within the path?

@jon-larsen
Copy link
Author

That indeed looks weird. Not sure what happened there, but I did a lot of trial and error when I first had this problem. Maybe there is something I have changed and not reverted...

Here's the current output. The firmware path is correct here:

TASK [Upload firmware through local file] **************************************
task path: /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/firmware-upload.yml:45
<osl-sw-tele-02> ESTABLISH LOCAL CONNECTION FOR USER: github
<osl-sw-tele-02> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/github/.***/tmp/***-local-119566qcdkk4dj `"&& mkdir "` echo /home/github/.***/tmp/***-local-119566qcdkk4dj/***-tmp-1709559405.8967009-119612-68289234745965 `" && echo ***-tmp-1709559405.8967009-119612-68289234745965="` echo /home/github/.***/tmp/***-local-119566qcdkk4dj/***-tmp-1709559405.8967009-119612-68289234745965 `" ) && sleep 0'
Using module file /home/github/.***/collections/***_collections/arubanetworks/aoscx/plugins/modules/aoscx_upload_firmware.py
<osl-sw-tele-02> PUT /home/github/.***/tmp/***-local-119566qcdkk4dj/tmp9hgjfq8w TO /home/github/.***/tmp/***-local-119566qcdkk4dj/***-tmp-1709559405.8967009-119612-68289234745965/AnsiballZ_aoscx_upload_firmware.py
<osl-sw-tele-02> EXEC /bin/sh -c 'chmod u+x /home/github/.***/tmp/***-local-119566qcdkk4dj/***-tmp-1709559405.8967009-119612-68289234745965/ /home/github/.***/tmp/***-local-119566qcdkk4dj/***-tmp-1709559405.8967009-119612-68289234745965/AnsiballZ_aoscx_upload_firmware.py && sleep 0'
<osl-sw-tele-02> EXEC /bin/sh -c '/usr/bin/python3 /home/github/.***/tmp/***-local-119566qcdkk4dj/***-tmp-1709559405.8967009-119612-68289234745965/AnsiballZ_aoscx_upload_firmware.py && sleep 0'
<osl-sw-tele-02> EXEC /bin/sh -c 'rm -f -r /home/github/.***/tmp/***-local-119566qcdkk4dj/***-tmp-1709559405.8967009-119612-68289234745965/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
  File "/tmp/***_arubanetworks.aoscx.aoscx_upload_firmware_payload_ls604x2i/***_arubanetworks.aoscx.aoscx_upload_firmware_payload.zip/***_collections/arubanetworks/aoscx/plugins/modules/aoscx_upload_firmware.py", line 149, in main
  File "/home/github/.local/lib/python3.8/site-packages/pyaoscx/session.py", line 165, in open
    raise LoginError("Cookies were not set correctly. Login failed")
fatal: [osl-sw-tele-02]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "firmware_file_path": "/data/firmware/ArubaOS-CX_6200_10_13_0005.swi",
            "partition_name": "secondary",
            "remote_firmware_file_path": null,
            "vrf": null,
            "wait_firmware_upload": false
        }
    },
    "msg": "Could not get PYAOSCX Session: 'LOGIN ERROR, Cookies were not set correctly. Login failed'"
}

@tchiapuziowong
Copy link
Member

since this is a login error can you verify if other modules in your environment present the same error or is it just the aoscx_upload_firmware - can you try aoscx_facts or aoscx_interface in a playbook and see if that is successful?

@tchiapuziowong
Copy link
Member

can you verify which version of the CX collection you have with the ansible-galaxy collection list command? Also within your inventory are you using an IP address or the FQDN to connect to the device?

@jon-larsen
Copy link
Author

Yes. It gets the current firmware first, to check if firmware upload is necessary:

TASK [Get current firmware version] ********************************************
task path: /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/firmware-upload.yml:11
<osl-sw-tele-02> ESTABLISH LOCAL CONNECTION FOR USER: github
<osl-sw-tele-02> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/github/.***/tmp/***-local-119566qcdkk4dj `"&& mkdir "` echo /home/github/.***/tmp/***-local-119566qcdkk4dj/***-tmp-1709559403.215948-119571-26433503054986 `" && echo ***-tmp-1709559403.215948-119571-26433503054986="` echo /home/github/.***/tmp/***-local-119566qcdkk4dj/***-tmp-1709559403.215948-119571-26433503054986 `" ) && sleep 0'
Using module file /home/github/.***/collections/***_collections/arubanetworks/aoscx/plugins/modules/aoscx_facts.py
<osl-sw-tele-02> PUT /home/github/.***/tmp/***-local-119566qcdkk4dj/tmp3_czjrv6 TO /home/github/.***/tmp/***-local-119566qcdkk4dj/***-tmp-1709559403.215948-119571-26433503054986/AnsiballZ_aoscx_facts.py
<osl-sw-tele-02> EXEC /bin/sh -c 'chmod u+x /home/github/.***/tmp/***-local-119566qcdkk4dj/***-tmp-1709559403.215948-119571-26433503054986/ /home/github/.***/tmp/***-local-119566qcdkk4dj/***-tmp-1709559403.215948-119571-26433503054986/AnsiballZ_aoscx_facts.py && sleep 0'
<osl-sw-tele-02> EXEC /bin/sh -c '/usr/bin/python3 /home/github/.***/tmp/***-local-119566qcdkk4dj/***-tmp-1709559403.215948-119571-26433503054986/AnsiballZ_aoscx_facts.py && sleep 0'
<osl-sw-tele-02> EXEC /bin/sh -c 'rm -f -r /home/github/.***/tmp/***-local-119566qcdkk4dj/***-tmp-1709559403.215948-119571-26433503054986/ > /dev/null 2>&1 && sleep 0'
ok: [osl-sw-tele-02] => {
    "***_facts": {
        "***_net_gather_network_resources": null,
        "***_net_gather_subset": [
            "software_version",
            "software_images",
            "management_interface",
            "software_version"
        ],
        "***_net_mgmt_intf_status": {
            "domain_name": "",
            "hostname": "osl-sw-tele-02",
            "link_state": "down",
            "mac_address": "b8:d4:e7:b5:7b:01"
        },
        "***_net_software_images": {
            "boot_profile_timeout": 5,
            "default_image": "primary",
            "primary_image_date": "2023-10-03 19:10:01 UTC",
            "primary_image_sha_256": "e9d39d3dba90b111faee4efcb279455a9c0f81d941dd0c38f97ec5a3640a9c60",
            "primary_image_size": "456 MB",
            "primary_image_version": "ML.10.10.1090",
            "secondary_image_date": "2023-10-03 19:10:01 UTC",
            "secondary_image_sha_256": "e9d39d3dba90b111faee4efcb279455a9c0f81d941dd0c38f97ec5a3640a9c60",
            "secondary_image_size": "456 MB",
            "secondary_image_version": "ML.10.10.1090"
        },
        "***_net_software_version": "ML.10.10.1090",
        "***_network_resources": {}
    },
    "changed": false,
    "invocation": {
        "module_args": {
            "gather_network_resources": null,
            "gather_subset": [
                "software_version",
                "software_images",
                "management_interface",
                "software_version"
            ]
        }
    }
}

Full list of collections:

# /home/github/.ansible/collections/ansible_collections
Collection               Version
------------------------ -------
ansible.netcommon        6.0.0  
ansible.utils            3.1.0  
arubanetworks.aos_switch 1.7.0  
arubanetworks.aoscx      4.3.2  

# /usr/lib/python3/dist-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    2.3.0  
ansible.netcommon             2.6.1  
ansible.posix                 1.4.0  
ansible.utils                 2.6.1  
ansible.windows               1.10.0 
arista.eos                    3.1.0  
awx.awx                       19.4.0 
azure.azcollection            1.13.0 
check_point.mgmt              2.3.0  
chocolatey.chocolatey         1.2.0  
cisco.aci                     2.2.0  
cisco.asa                     2.1.0  
cisco.dnac                    6.5.0  
cisco.intersight              1.0.19 
cisco.ios                     2.8.1  
cisco.iosxr                   2.9.0  
cisco.ise                     1.2.1  
cisco.meraki                  2.8.0  
cisco.mso                     1.4.0  
cisco.nso                     1.0.3  
cisco.nxos                    2.9.1  
cisco.ucs                     1.8.0  
cloud.common                  2.1.2  
cloudscale_ch.cloud           2.2.2  
community.aws                 2.6.1  
community.azure               1.1.0  
community.ciscosmb            1.0.5  
community.crypto              2.3.4  
community.digitalocean        1.20.0 
community.dns                 2.2.0  
community.docker              2.6.0  
community.fortios             1.0.0  
community.general             4.8.3  
community.google              1.0.0  
community.grafana             1.5.0  
community.hashi_vault         2.5.0  
community.hrobot              1.4.0  
community.kubernetes          2.0.1  
community.kubevirt            1.0.0  
community.libvirt             1.1.0  
community.mongodb             1.4.1  
community.mysql               2.3.8  
community.network             3.3.0  
community.okd                 2.2.0  
community.postgresql          1.7.4  
community.proxysql            1.4.0  
community.rabbitmq            1.2.1  
community.routeros            2.1.0  
community.sap                 1.0.0  
community.sap_libs            1.1.0  
community.skydive             1.0.0  
community.sops                1.2.2  
community.vmware              1.18.2 
community.windows             1.10.0 
community.zabbix              1.7.0  
containers.podman             1.9.3  
cyberark.conjur               1.1.0  
cyberark.pas                  1.0.14 
dellemc.enterprise_sonic      1.1.1  
dellemc.openmanage            4.4.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.17.0 
fortinet.fortimanager         2.1.5  
fortinet.fortios              2.1.6  
frr.frr                       1.0.4  
gluster.gluster               1.0.2  
google.cloud                  1.0.2  
hetzner.hcloud                1.6.0  
hpe.nimble                    1.1.4  
ibm.qradar                    1.0.3  
infinidat.infinibox           1.3.3  
infoblox.nios_modules         1.2.2  
inspur.sm                     1.3.0  
junipernetworks.junos         2.10.0 
kubernetes.core               2.3.2  
mellanox.onyx                 1.0.0  
netapp.aws                    21.7.0 
netapp.azure                  21.10.0
netapp.cloudmanager           21.18.0
netapp.elementsw              21.7.0 
netapp.ontap                  21.20.0
netapp.storagegrid            21.10.0
netapp.um_info                21.8.0 
netapp_eseries.santricity     1.3.0  
netbox.netbox                 3.7.1  
ngine_io.cloudstack           2.2.4  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.2  
openstack.cloud               1.8.0  
openvswitch.openvswitch       2.1.0  
ovirt.ovirt                   1.6.6  
purestorage.flasharray        1.13.0 
purestorage.flashblade        1.9.0  
sensu.sensu_go                1.13.1 
servicenow.servicenow         1.0.6  
splunk.es                     1.0.2  
t_systems_mms.icinga_director 1.30.0 
theforeman.foreman            2.2.0  
vmware.vmware_rest            2.2.0  
vyos.vyos                     2.8.0  
wti.remote                    1.0.4

And Python packages:

$ pip3 list
Package                Version             
---------------------- --------------------
ansible                5.10.0              
ansible-core           2.12.10             
ansible-pylibssh       0.2.0               
attrs                  19.3.0              
Automat                0.8.0               
bcrypt                 3.2.0               
blinker                1.4                 
certifi                2019.11.28          
certsrv                2.1.1               
cffi                   1.14.6              
chardet                3.0.4               
Click                  7.0                 
cloud-init             23.4.4              
colorama               0.4.3               
command-not-found      0.3                 
configobj              5.0.6               
constantly             15.1.0              
cryptography           2.8                 
dbus-python            1.2.16              
distro                 1.4.0               
distro-info            0.23+ubuntu1.1      
entrypoints            0.3                 
httplib2               0.14.0              
hyperlink              19.0.0              
idna                   2.8                 
importlib-metadata     1.5.0               
incremental            16.10.1             
Jinja2                 2.10.1              
jmespath               0.9.4               
jsonpatch              1.22                
jsonpointer            2.0                 
jsonschema             3.2.0               
jxmlease               1.0.3               
keyring                18.0.1              
language-selector      0.1                 
launchpadlib           1.10.13             
lazr.restfulclient     0.14.2              
lazr.uri               1.0.3               
lxml                   4.6.3               
MarkupSafe             1.1.0               
more-itertools         4.2.0               
ncclient               0.6.12              
netaddr                0.8.0               
netifaces              0.10.4              
ntlm-auth              1.1.0               
oauthlib               3.1.0               
packaging              20.3                
paramiko               2.7.2               
pexpect                4.6.0               
pip                    20.0.2              
pyaoscx                2.5.1               
pyasn1                 0.4.2               
pyasn1-modules         0.2.1               
pycparser              2.20                
pycrypto               2.6.1               
pycurl                 7.45.2              
PyGObject              3.36.0              
PyHamcrest             1.9.0               
PyJWT                  1.7.1               
pykerberos             1.1.14              
pymacaroons            0.13.0              
PyNaCl                 1.3.0               
pyOpenSSL              19.0.0              
pyparsing              2.4.6               
pyrsistent             0.15.5              
pyserial               3.4                 
python-apt             2.0.1+ubuntu0.20.4.1
python-debian          0.1.36+ubuntu1.1    
pywinrm                0.3.0               
PyYAML                 5.3.1               
requests               2.22.0              
requests-kerberos      0.12.0              
requests-ntlm          1.1.0               
requests-toolbelt      1.0.0               
requests-unixsocket    0.2.0               
resolvelib             0.5.4               
SecretStorage          2.3.1               
service-identity       18.1.0              
setuptools             45.2.0              
simplejson             3.16.0              
six                    1.14.0              
sos                    4.5.6               
ssh-import-id          5.10                
systemd-python         234                 
Twisted                18.9.0              
ubuntu-advantage-tools 8001                
ufw                    0.36                
unattended-upgrades    0.1                 
urllib3                1.25.8              
wadllib                1.3.3               
wheel                  0.34.2              
xmltodict              0.12.0              
zipp                   1.0.0               
zope.interface         4.7.1

@tchiapuziowong
Copy link
Member

@jon-larsen can you confirm if you're still experiencing this issue with the latest collection and pyaoscx version?

@jon-larsen
Copy link
Author

jon-larsen commented Jul 23, 2024 •
edited
Loading

I still get the problem with the following versions:

# /home/github/.ansible/collections/ansible_collections
Collection               Version
------------------------ -------
ansible.netcommon        5.3.0  
ansible.utils            2.12.0 
arubanetworks.aos_switch 1.7.0  
arubanetworks.aoscx      4.4.0  

$ pip3 list | grep pyaoscx
pyaoscx             2.6.0

We have however changed from ansible-core to ansible a while back, and now I see the following warning in the play:
Warning: : Collection arubanetworks.aoscx does not support Ansible version

# ansible --version
ansible 2.10.8
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]

Running the default Ansible version in Ubuntu 22.04.

@tchiapuziowong
Copy link
Member

@jon-larsen thanks for the clarification - I'm thinking this is due to Ansible using an out of date version of the pyaoscx package - I'm noticing some inconsistencies with the error & the details of your environment - could you please verify if the python package of pyaoscx is the latest that ansible is using?

I noticed in the previous error message the error was being thrown at line "/home/github/.local/lib/python3.8/site-packages/pyaoscx/session.py", line 165 but in your Ansible details it seems like the python version is 3.10 with module location /usr/lib/python3/ - as well as in the current pyaoscx version, line 165 is not the raise LoginError as it was in version 2.4.1

Can you execute the following commands and supply the output? Also please supply the output of the error if it's different/changed from when previously posted
which pip3
which python3
which ansible

@jon-larsen
Copy link
Author

Thanks Tiffany, that is strange indeed. I wonder if I should redeploy the VM that we use for Ansible, as we've been doing lots of manual configuration on it during troubleshooting of all different kinds of issues with Ansible.

We have the following configuration in /etc/ansible/ansible.cfg:

interpreter_python=/usr/bin/python3

Here's the output of which:

# su - github
$ which pip3
/home/github/.local/bin/pip3
$ which python3
/usr/bin/python3
$ which ansible
/usr/bin/ansible

With the latest run, it looks like it is using python 3.10 and the correct path for pyaoscx. The date on the file looks correct at least:

ls -l /home/github/.local/lib/python3.10/site-packages/pyaoscx/session.py
-rw-rw-r-- 1 github github 12630 Jul 23 10:35 /home/github/.local/lib/python3.10/site-packages/pyaoscx/session.py
Here is the latest output of the playbook-run 
***-playbook 2.10.8
  config file = /etc/***/***.cfg
  configured module search path = ['/home/github/.***/plugins/modules', '/usr/share/***/plugins/modules']
  *** python module location = /usr/lib/python3/dist-packages/***
  executable location = /usr/bin/***-playbook
  python version = 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
Using /etc/***/***.cfg as config file
host_list declined parsing /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/inventory as it did not pass its verify_file() method
script declined parsing /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/inventory as it did not pass its verify_file() method
auto declined parsing /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/inventory as it did not pass its verify_file() method
Parsed /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/inventory inventory source with yaml plugin
Warning: : Collection arubanetworks.aoscx does not support Ansible version
2.10.8
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: firmware-upload.yml **************************************************
1 plays in firmware-upload.yml

PLAY [osl-sw-tele-02] **********************************************************
META: ran handlers

TASK [Get current firmware version] ********************************************
task path: /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/firmware-upload.yml:10
<osl-sw-tele-02> ESTABLISH LOCAL CONNECTION FOR USER: github
<osl-sw-tele-02> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/github/.***/tmp/***-local-132191lllh_6q3 `"&& mkdir "` echo /home/github/.***/tmp/***-local-132191lllh_6q3/***-tmp-1721724613.972546-132195-62433496564508 `" && echo ***-tmp-1721724613.972546-132195-62433496564508="` echo /home/github/.***/tmp/***-local-132191lllh_6q3/***-tmp-1721724613.972546-132195-62433496564508 `" ) && sleep 0'
Using module file /home/github/.***/collections/***_collections/arubanetworks/aoscx/plugins/modules/aoscx_facts.py
<osl-sw-tele-02> PUT /home/github/.***/tmp/***-local-132191lllh_6q3/tmp8q4ittol TO /home/github/.***/tmp/***-local-132191lllh_6q3/***-tmp-1721724613.972546-132195-62433496564508/AnsiballZ_aoscx_facts.py
<osl-sw-tele-02> EXEC /bin/sh -c 'chmod u+x /home/github/.***/tmp/***-local-132191lllh_6q3/***-tmp-1721724613.972546-132195-62433496564508/ /home/github/.***/tmp/***-local-132191lllh_6q3/***-tmp-1721724613.972546-132195-62433496564508/AnsiballZ_aoscx_facts.py && sleep 0'
<osl-sw-tele-02> EXEC /bin/sh -c '/usr/bin/python3 /home/github/.***/tmp/***-local-132191lllh_6q3/***-tmp-1721724613.972546-132195-62433496564508/AnsiballZ_aoscx_facts.py && sleep 0'
<osl-sw-tele-02> EXEC /bin/sh -c 'rm -f -r /home/github/.***/tmp/***-local-132191lllh_6q3/***-tmp-1721724613.972546-132195-62433496564508/ > /dev/null 2>&1 && sleep 0'
ok: [osl-sw-tele-02] => {
    "***_facts": {
        "***_net_gather_network_resources": null,
        "***_net_gather_subset": [
            "software_version",
            "software_images",
            "management_interface",
            "software_version"
        ],
        "***_net_mgmt_intf_status": {
            "domain_name": "",
            "hostname": "osl-sw-tele-02",
            "link_state": "down",
            "mac_address": "b8:d4:e7:b5:7b:01"
        },
        "***_net_software_images": {
            "boot_profile_timeout": 5,
            "default_image": "secondary",
            "primary_image_date": "2024-01-29 21:00:16 UTC",
            "primary_image_sha_256": "235c516e5044c31ed3ae1049463bdb86803779b741513f74ade249fa8566c30d",
            "primary_image_size": "606 MB",
            "primary_image_version": "ML.10.13.1000",
            "secondary_image_date": "2024-05-16 17:49:59 UTC",
            "secondary_image_sha_256": "9cb1edbf80cc49ec44f9fe684f374fe1a2ce5667854d559eaaf0c35bd4ac23a8",
            "secondary_image_size": "405 MB",
            "secondary_image_version": "ML.10.13.1020"
        },
        "***_net_software_version": "ML.10.13.1020",
        "***_network_resources": {}
    },
    "changed": false,
    "invocation": {
        "module_args": {
            "gather_network_resources": null,
            "gather_subset": [
                "software_version",
                "software_images",
                "management_interface",
                "software_version"
            ]
        }
    }
}

TASK [Set current_version variable] ********************************************
task path: /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/firmware-upload.yml:15
ok: [osl-sw-tele-02] => {
    "***_facts": {
        "current_version": "ML.10.13.1020"
    },
    "changed": false
}

TASK [Set image_version variable] **********************************************
task path: /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/firmware-upload.yml:19
ok: [osl-sw-tele-02] => {
    "***_facts": {
        "image_version": "ML.10.13.1000"
    },
    "changed": false
}

TASK [Upload firmware through local file] **************************************
task path: /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/firmware-upload.yml:44
<osl-sw-tele-02> ESTABLISH LOCAL CONNECTION FOR USER: github
<osl-sw-tele-02> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/github/.***/tmp/***-local-132191lllh_6q3 `"&& mkdir "` echo /home/github/.***/tmp/***-local-132191lllh_6q3/***-tmp-1721724618.9154713-132233-142907318771440 `" && echo ***-tmp-1721724618.9154713-132233-142907318771440="` echo /home/github/.***/tmp/***-local-132191lllh_6q3/***-tmp-1721724618.9154713-132233-142907318771440 `" ) && sleep 0'
Using module file /home/github/.***/collections/***_collections/arubanetworks/aoscx/plugins/modules/aoscx_upload_firmware.py
<osl-sw-tele-02> PUT /home/github/.***/tmp/***-local-132191lllh_6q3/tmpzp96xrg0 TO /home/github/.***/tmp/***-local-132191lllh_6q3/***-tmp-1721724618.9154713-132233-142907318771440/AnsiballZ_aoscx_upload_firmware.py
<osl-sw-tele-02> EXEC /bin/sh -c 'chmod u+x /home/github/.***/tmp/***-local-132191lllh_6q3/***-tmp-1721724618.9154713-132233-142907318771440/ /home/github/.***/tmp/***-local-132191lllh_6q3/***-tmp-1721724618.9154713-132233-142907318771440/AnsiballZ_aoscx_upload_firmware.py && sleep 0'
<osl-sw-tele-02> EXEC /bin/sh -c '/usr/bin/python3 /home/github/.***/tmp/***-local-132191lllh_6q3/***-tmp-1721724618.9154713-132233-142907318771440/AnsiballZ_aoscx_upload_firmware.py && sleep 0'
<osl-sw-tele-02> EXEC /bin/sh -c 'rm -f -r /home/github/.***/tmp/***-local-132191lllh_6q3/***-tmp-1721724618.9154713-132233-142907318771440/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
  File "/tmp/***_arubanetworks.aoscx.aoscx_upload_firmware_payload_pdbjkbwx/***_arubanetworks.aoscx.aoscx_upload_firmware_payload.zip/***_collections/arubanetworks/aoscx/plugins/modules/aoscx_upload_firmware.py", line 149, in main
  File "/home/github/.local/lib/python3.10/site-packages/pyaoscx/session.py", line 173, in open
    raise LoginError("Cookies were not set correctly. Login failed")
fatal: [osl-sw-tele-02]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "firmware_file_path": "/data/firmware/ArubaOS-CX_6200_10_13_1031.swi",
            "partition_name": "primary",
            "remote_firmware_file_path": null,
            "vrf": null,
            "wait_firmware_upload": false
        }
    },
    "msg": "Could not get PYAOSCX Session: 'LOGIN ERROR, Cookies were not set correctly. Login failed'"
}

PLAY RECAP *********************************************************************
osl-sw-tele-02             : ok=3    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

@jon-larsen
Copy link
Author

I redeployed the VM (now with Ubuntu 24.04), but still the same problem.
PS: Now I also encountered #99 , which was not an issue before. I worked around this problem as described in the issue.

Here is the output of the run from the new Ansible VM 
***-playbook [core 2.16.3]
  config file = None
  configured module search path = ['/home/github/.***/plugins/modules', '/usr/share/***/plugins/modules']
  *** python module location = /usr/lib/python3/dist-packages/***
  *** collection location = /home/github/.***/collections:/usr/share/***/collections
  executable location = /usr/bin/***-playbook
  python version = 3.12.3 (main, Apr 10 2024, 05:33:47) [GCC 13.2.0] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True
No config file found; using defaults
host_list declined parsing /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/inventory as it did not pass its verify_file() method
script declined parsing /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/inventory as it did not pass its verify_file() method
auto declined parsing /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/inventory as it did not pass its verify_file() method
Parsed /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/inventory inventory source with yaml plugin
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.
PLAYBOOK: firmware-upload.yml **************************************************
1 plays in firmware-upload.yml
PLAY [osl-sw-tele-02] **********************************************************
TASK [Get current firmware version] ********************************************
task path: /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/firmware-upload.yml:10
<osl-sw-tele-02> ESTABLISH LOCAL CONNECTION FOR USER: github
<osl-sw-tele-02> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/github/.***/tmp/***-local-32856lb2rqkvr `"&& mkdir "` echo /home/github/.***/tmp/***-local-32856lb2rqkvr/***-tmp-1721828367.3330443-32860-230888377649594 `" && echo ***-tmp-1721828367.3330443-32860-230888377649594="` echo /home/github/.***/tmp/***-local-32856lb2rqkvr/***-tmp-1721828367.3330443-32860-230888377649594 `" ) && sleep 0'
Using module file /home/github/.***/collections/***_collections/arubanetworks/aoscx/plugins/modules/aoscx_facts.py
<osl-sw-tele-02> PUT /home/github/.***/tmp/***-local-32856lb2rqkvr/tmpfbsvelc5 TO /home/github/.***/tmp/***-local-32856lb2rqkvr/***-tmp-1721828367.3330443-32860-230888377649594/AnsiballZ_aoscx_facts.py
<osl-sw-tele-02> EXEC /bin/sh -c 'chmod u+x /home/github/.***/tmp/***-local-32856lb2rqkvr/***-tmp-1721828367.3330443-32860-230888377649594/ /home/github/.***/tmp/***-local-32856lb2rqkvr/***-tmp-1721828367.3330443-32860-230888377649594/AnsiballZ_aoscx_facts.py && sleep 0'
<osl-sw-tele-02> EXEC /bin/sh -c '/usr/bin/python3 /home/github/.***/tmp/***-local-32856lb2rqkvr/***-tmp-1721828367.3330443-32860-230888377649594/AnsiballZ_aoscx_facts.py && sleep 0'
<osl-sw-tele-02> EXEC /bin/sh -c 'rm -f -r /home/github/.***/tmp/***-local-32856lb2rqkvr/***-tmp-1721828367.3330443-32860-230888377649594/ > /dev/null 2>&1 && sleep 0'
ok: [osl-sw-tele-02] => {
    "***_facts": {
        "***_net_gather_network_resources": null,
        "***_net_gather_subset": [
            "software_version",
            "software_images",
            "management_interface",
            "software_version"
        ],
        "***_net_mgmt_intf_status": {
            "domain_name": "",
            "hostname": "osl-sw-tele-02",
            "link_state": "down",
            "mac_address": "b8:d4:e7:b5:7b:01"
        },
        "***_net_software_images": {
            "boot_profile_timeout": 5,
            "default_image": "secondary",
            "primary_image_date": "2024-01-29 21:00:16 UTC",
            "primary_image_sha_256": "235c516e5044c31ed3ae1049463bdb86803779b741513f74ade249fa8566c30d",
            "primary_image_size": "606 MB",
            "primary_image_version": "ML.10.13.1000",
            "secondary_image_date": "2024-05-16 17:49:59 UTC",
            "secondary_image_sha_256": "9cb1edbf80cc49ec44f9fe684f374fe1a2ce5667854d559eaaf0c35bd4ac23a8",
            "secondary_image_size": "405 MB",
            "secondary_image_version": "ML.10.13.1020"
        },
        "***_net_software_version": "ML.10.13.1020",
        "***_network_resources": {}
    },
    "changed": false,
    "invocation": {
        "module_args": {
            "gather_network_resources": null,
            "gather_subset": [
                "software_version",
                "software_images",
                "management_interface",
                "software_version"
            ]
        }
    }
}
TASK [Set current_version variable] ********************************************
task path: /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/firmware-upload.yml:15
ok: [osl-sw-tele-02] => {
    "***_facts": {
        "current_version": "ML.10.13.1020"
    },
    "changed": false
}
TASK [Set image_version variable] **********************************************
task path: /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/firmware-upload.yml:19
ok: [osl-sw-tele-02] => {
    "***_facts": {
        "image_version": "ML.10.13.1000"
    },
    "changed": false
}
TASK [Upload firmware through local file] **************************************
task path: /data/github-runner/_work/drift-nettverk-switch-firmware/drift-nettverk-switch-firmware/firmware-upload.yml:44
<osl-sw-tele-02> ESTABLISH LOCAL CONNECTION FOR USER: github
<osl-sw-tele-02> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/github/.***/tmp/***-local-32856lb2rqkvr `"&& mkdir "` echo /home/github/.***/tmp/***-local-32856lb2rqkvr/***-tmp-1721828370.919814-32890-186265003618712 `" && echo ***-tmp-1721828370.919814-32890-186265003618712="` echo /home/github/.***/tmp/***-local-32856lb2rqkvr/***-tmp-1721828370.919814-32890-186265003618712 `" ) && sleep 0'
Using module file /home/github/.***/collections/***_collections/arubanetworks/aoscx/plugins/modules/aoscx_upload_firmware.py
<osl-sw-tele-02> PUT /home/github/.***/tmp/***-local-32856lb2rqkvr/tmp1dwi3h0q TO /home/github/.***/tmp/***-local-32856lb2rqkvr/***-tmp-1721828370.919814-32890-186265003618712/AnsiballZ_aoscx_upload_firmware.py
<osl-sw-tele-02> EXEC /bin/sh -c 'chmod u+x /home/github/.***/tmp/***-local-32856lb2rqkvr/***-tmp-1721828370.919814-32890-186265003618712/ /home/github/.***/tmp/***-local-32856lb2rqkvr/***-tmp-1721828370.919814-32890-186265003618712/AnsiballZ_aoscx_upload_firmware.py && sleep 0'
<osl-sw-tele-02> EXEC /bin/sh -c '/usr/bin/python3 /home/github/.***/tmp/***-local-32856lb2rqkvr/***-tmp-1721828370.919814-32890-186265003618712/AnsiballZ_aoscx_upload_firmware.py && sleep 0'
<osl-sw-tele-02> EXEC /bin/sh -c 'rm -f -r /home/github/.***/tmp/***-local-32856lb2rqkvr/***-tmp-1721828370.919814-32890-186265003618712/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
  File "/tmp/***_arubanetworks.aoscx.aoscx_upload_firmware_payload_9v2an07a/***_arubanetworks.aoscx.aoscx_upload_firmware_payload.zip/***_collections/arubanetworks/aoscx/plugins/modules/aoscx_upload_firmware.py", line 149, in main
    session.open(
  File "/home/github/.local/lib/python3.12/site-packages/pyaoscx/session.py", line 173, in open
    raise LoginError("Cookies were not set correctly. Login failed")
fatal: [osl-sw-tele-02]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "firmware_file_path": "/data/firmware/ArubaOS-CX_6200_10_13_1031.swi",
            "partition_name": "primary",
            "remote_firmware_file_path": null,
            "vrf": null,
            "wait_firmware_upload": false
        }
    },
    "msg": "Could not get PYAOSCX Session: 'LOGIN ERROR, Cookies were not set correctly. Login failed'"
}
PLAY RECAP *********************************************************************
osl-sw-tele-02             : ok=3    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

$ ansible-galaxy collection list

# /home/github/.ansible/collections/ansible_collections
Collection                               Version
---------------------------------------- -------
ansible.netcommon                        5.3.0  
ansible.utils                            2.12.0 
arubanetworks.aos_switch                 1.7.0  
arubanetworks.aoscx                      4.4.0  

$ pip3 list | grep pyaoscx
pyaoscx             2.6.0

$ which pip3
/usr/bin/pip3
$ which python3
/usr/bin/python3
$ which ansible
/usr/bin/ansible

@HansM1986
Copy link

HansM1986 commented Dec 11, 2024 •
edited
Loading

I am also having similar issues (LOGIN ERROR, Cookies where not set correctly. Login failed).

I managed to solve the Login failed error by replacing the following lines in the aoscx_upload_firmware.py:

# diff aoscx_upload_firmware.py aoscx_upload_firmware.py.backup
142a143
>         proxy = session.proxy.get("https", None)
148c149,153
<         session = get_pyaoscx_session(ansible_module)
---
>         session.open(
>             username=session.username(),
>             password=session.password(),
>             use_proxy=proxy is not None,
>         )

Unfortunately, this only lead to new errors:

The full traceback is:
Traceback (most recent call last):
  File "/usr/share/ansible/.tmp/.ansible/tmp/ansible-local-5967dtabyo5/ansible-tmp-1733905547.0817366-618-108781600818647/AnsiballZ_aoscx_upload_firmware.py", line 107, in <module>
    _ansiballz_main()
  File "/usr/share/ansible/.tmp/.ansible/tmp/ansible-local-5967dtabyo5/ansible-tmp-1733905547.0817366-618-108781600818647/AnsiballZ_aoscx_upload_firmware.py", line 99, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/usr/share/ansible/.tmp/.ansible/tmp/ansible-local-5967dtabyo5/ansible-tmp-1733905547.0817366-618-108781600818647/AnsiballZ_aoscx_upload_firmware.py", line 47, in invoke_module
    runpy.run_module(mod_name='ansible_collections.arubanetworks.aoscx.plugins.modules.aoscx_upload_firmware', init_globals=dict(_module_fqn='ansible_collections.arubanetworks.aoscx.plugins.modules.aoscx_upload_firmware', _modlib_path=modlib_path),
  File "<frozen runpy>", line 226, in run_module
  File "<frozen runpy>", line 98, in _run_module_code
  File "<frozen runpy>", line 88, in _run_code
  File "/tmp/ansible_arubanetworks.aoscx.aoscx_upload_firmware_payload_7p39h4xv/ansible_arubanetworks.aoscx.aoscx_upload_firmware_payload.zip/ansible_collections/arubanetworks/aoscx/plugins/modules/aoscx_upload_firmware.py", line 218, in <module>
  File "/tmp/ansible_arubanetworks.aoscx.aoscx_upload_firmware_payload_7p39h4xv/ansible_arubanetworks.aoscx.aoscx_upload_firmware_payload.zip/ansible_collections/arubanetworks/aoscx/plugins/modules/aoscx_upload_firmware.py", line 154, in main
  File "/var/lib/semaphore/lib/python3.12/site-packages/pyaoscx/pyaoscx_factory.py", line 22, in __call__
    cls.__instance = super(Singleton, cls).__call__(*args, **kwargs)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/semaphore/lib/python3.12/site-packages/pyaoscx/device.py", line 36, in __init__
    self.get_firmware_version()
  File "/var/lib/semaphore/lib/python3.12/site-packages/pyaoscx/pyaoscx_module.py", line 40, in ensure_connected
    return fnct(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/semaphore/lib/python3.12/site-packages/pyaoscx/device.py", line 190, in get_firmware_version
    data = self.get_firmware_info()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/semaphore/lib/python3.12/site-packages/pyaoscx/pyaoscx_module.py", line 40, in ensure_connected
    return fnct(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/semaphore/lib/python3.12/site-packages/pyaoscx/device.py", line 210, in get_firmware_info
    raise GenericOperationError(response.text, response.status_code)
pyaoscx.exceptions.generic_op_error.GenericOperationError: 'GENERIC OPERATION ERROR: Unauthorized: Code: 401'

I'm running out of time to look into this issue, so I just wanted to share my findings in case they might be usefull for anyone.

For the record, I'm running this in an Alpine 3.20 docker image.

Details 

d3805d264054:~$ ansible --version
ansible [core 2.18.1]
  config file = None
  configured module search path = ['/home/semaphore/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /var/lib/semaphore/lib/python3.12/site-packages/ansible
  ansible collection location = /home/semaphore/.ansible/collections:/usr/share/ansible/collections
  executable location = /var/lib/semaphore/bin/ansible
  python version = 3.12.8 (main, Dec  7 2024, 00:01:15) [GCC 13.2.1 20240309] (/var/lib/semaphore/bin/python)
  jinja version = 3.1.4
  libyaml = True


d3805d264054:~$ pip list
Package            Version
------------------ -----------
ansible            11.1.0
ansible-core       2.18.1
ansible-pylibssh   1.2.2
bcrypt             4.2.1
boto3              1.35.77
botocore           1.35.77
certifi            2024.2.2
cffi               1.17.1
charset-normalizer 3.3.2
cryptography       44.0.0
docker             7.1.0
editables          0.5
grpcio             1.68.1
hatch-vcs          0.4.0
hatchling          1.24.2
idna               3.7
Jinja2             3.1.4
jmespath           1.0.1
jxmlease           1.0.3
lxml               5.3.0
MarkupSafe         3.0.2
ncclient           0.6.16
netaddr            1.3.0
packaging          24.0
paramiko           3.5.0
passlib            1.7.4
pathspec           0.12.1
pip                24.3.1
pluggy             1.5.0
protobuf           5.29.1
pyaoscx            2.6.0
pycparser          2.22
pycurl             7.45.3
PyNaCl             1.5.0
pyparsing          3.1.2
pyspnego           0.11.2
python-dateutil    2.9.0.post0
pywinrm            0.5.0
PyYAML             6.0.2
requests           2.32.3
requests_ntlm      1.3.0
requests-toolbelt  1.0.0
resolvelib         1.0.1
s3transfer         0.10.4
setuptools         70.3.0
setuptools-scm     8.0.4
six                1.17.0
trove-classifiers  2024.5.17
urllib3            1.26.18
websocket-client   1.8.0
wheel              0.45.1
xmltodict          0.14.2


d3805d264054:~$ ansible-galaxy collection list

# /usr/share/ansible/collections/ansible_collections
Collection                               Version
---------------------------------------- -------
arubanetworks.aoscx                      4.4.0
devsec.hardening                         9.0.0

# /var/lib/semaphore/lib/python3.12/site-packages/ansible_collections
Collection                               Version
---------------------------------------- -------
amazon.aws                               9.0.0
ansible.netcommon                        7.1.0
ansible.posix                            1.6.2
ansible.utils                            5.1.2
ansible.windows                          2.5.0
arista.eos                               10.0.1
awx.awx                                  24.6.1
azure.azcollection                       3.1.0
check_point.mgmt                         6.2.1
chocolatey.chocolatey                    1.5.3
cisco.aci                                2.10.1
cisco.asa                                6.0.0
cisco.dnac                               6.25.0
cisco.intersight                         2.0.20
cisco.ios                                9.0.3
cisco.iosxr                              10.2.2
cisco.ise                                2.9.6
cisco.meraki                             2.18.3
cisco.mso                                2.9.0
cisco.nxos                               9.2.1
cisco.ucs                                1.14.0
cloud.common                             4.0.0
cloudscale_ch.cloud                      2.4.0
community.aws                            9.0.0
community.ciscosmb                       1.0.9
community.crypto                         2.22.3
community.digitalocean                   1.27.0
community.dns                            3.1.0
community.docker                         4.1.0
community.general                        10.1.0
community.grafana                        2.1.0
community.hashi_vault                    6.2.0
community.hrobot                         2.0.2
community.library_inventory_filtering_v1 1.0.2
community.libvirt                        1.3.0
community.mongodb                        1.7.8
community.mysql                          3.11.0
community.network                        5.1.0
community.okd                            4.0.0
community.postgresql                     3.9.0
community.proxysql                       1.6.0
community.rabbitmq                       1.3.0
community.routeros                       3.1.0
community.sap_libs                       1.4.2
community.sops                           2.0.0
community.vmware                         5.2.0
community.windows                        2.3.0
community.zabbix                         3.2.0
containers.podman                        1.16.2
cyberark.conjur                          1.3.1
cyberark.pas                             1.0.30
dellemc.enterprise_sonic                 2.5.1
dellemc.openmanage                       9.9.0
dellemc.powerflex                        2.5.0
dellemc.unity                            2.0.0
f5networks.f5_modules                    1.32.1
fortinet.fortimanager                    2.8.2
fortinet.fortios                         2.3.8
google.cloud                             1.4.1
grafana.grafana                          5.6.0
hetzner.hcloud                           4.2.2
ibm.qradar                               4.0.0
ibm.spectrum_virtualize                  2.0.0
ibm.storage_virtualize                   2.5.0
ieisystem.inmanage                       3.0.0
infinidat.infinibox                      1.4.5
infoblox.nios_modules                    1.7.1
inspur.ispim                             2.2.3
junipernetworks.junos                    9.1.0
kaytus.ksmanage                          2.0.0
kubernetes.core                          5.0.0
kubevirt.core                            2.1.0
lowlydba.sqlserver                       2.3.4
microsoft.ad                             1.7.1
netapp.cloudmanager                      21.24.0
netapp.ontap                             22.13.0
netapp.storagegrid                       21.13.0
netapp_eseries.santricity                1.4.1
netbox.netbox                            3.20.0
ngine_io.cloudstack                      2.5.0
openstack.cloud                          2.3.0
ovirt.ovirt                              3.2.0
purestorage.flasharray                   1.32.0
purestorage.flashblade                   1.19.1
sensu.sensu_go                           1.14.0
splunk.es                                4.0.0
telekom_mms.icinga_director              2.2.1
theforeman.foreman                       4.2.0
vmware.vmware                            1.7.1
vmware.vmware_rest                       4.3.0
vultr.cloud                              1.13.0
vyos.vyos                                5.0.0
wti.remote                               1.0.10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tchiapuziowong tchiapuziowong

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@HansM1986 @tchiapuziowong @jon-larsen