mgmt interface with c8000v

[nagraj-kulkarni]

Hi, having hard time getting the connectivity from cisco 8000v to external world. Understand that the mgmt interface i.e first interface , gets qemu dhcp 10.x.x.x and it gets discovered by external routers etc. But i am unable to get the external reachability from the 8000v router itself, tried using the mgmt,addressing schemes in topology/static routes but not helping. How do get it to ping say 8.8.8.8 from the router ? appreciate any help :)

[ipspace]

The management interface (the first Ethernet interface) of Cisco 8000v (and most other devices) is in the management VRF. It should get the IP address from the libvirt DHCP server and it should be in the 192.168.121.0/24 range. To reach the outside world through the management VRF, you have to use the VRF version of the commands (ping vrf or similar)

You can also connect one of the data-plane interfaces to the outside world. See https://netlab.tools/labs/libvirt/#connecting-to-the-outside-world for details (assuming you're using libvirt because you mentioned Qemu)

[nagraj-kulkarni]

Hello, i am using the clab provider and the first interface is getting ip in the subnet 10.0.0.x from qemu dhcp. There is no vrf configured.
I used "addressing/mgmt" etc in the topology file so that it gets the first interface ip from these subnets ( 192.x). netlab status shows the managment ip as 192.x network, but the show running-config in the router does not show any interface having ip in 192.x network,so i believe clab is doing a magic in the underlying bridge, to get reachaibility to router. So in bried, I can do ssh to the router using 192.x, but no interface has ip configured in 192.x ( as per show running), ping from router to external machine (where i am running netlab) does not work, i can not give the default route as 192.x.x.1 in the router as this network is unknown :)

[ssasso]

When using containerlab, with the vrnetlab containers, a sort of 1:1 NAT is performed in userspace by qemu.

You will find an interface with 10.0.0.something, and the default gateway for this is 10.0.0.2

[ipspace]

Ah, vrnetlab. I never understood why someone would use that when there's a perfectly fine way to build virtual machine Vagrant boxes without containers. The convoluted networking they had to do to get things working led me to write this in https://netlab.tools/labs/clab/#using-vrnetlab-containers

vrnetlab is an independent open-source project. If it fails to produce a working container image (example), please contact them.

Anyway, vrnetlab got much better after @hellt forked it. At the very minimum, you should use the latest vrnetlab code from https://github.com/hellt/vrnetlab/ to build the container. For example, https://github.com/hellt/vrnetlab/tree/master/c8000v/docker puts the management interface into a VRF and configures the default route for the management VRF.

[hellt]

It would be assigned by container runtime in this case. Meaning you don't have to statically type it in, but you could assign it statically as well

From the node config perspective this would be a static configuration of the mgmt IP

[nagraj-kulkarni]

@ipspace @ssasso is there a way to create vagrant box without bringing down other labs. I have few labs setup and dont want to disturb them. Getting below error with netlab libvirt package csr disk-image:

netlab cannot create a Vagrant box while there are running labs. Inspect the
status of your lab instances with "netlab status" and stop them with
"netlab down" or "netlab status cleanup --all"

[ipspace]

Unfortunately not. Keep working with the setup you have right now, and build that box once you have to shutdown or reload the server.

[ssasso]

Unfortunately not. Keep working with the setup you have right now, and build that box once you have to shutdown or reload the server.

Or create a virtual machine with nested virtualization capabilities just to build the Vagrant box ;)

(even on some public cloud, most probably it should take less than an hour to build the vagrant box, so in theory the cost should be affordable)

[nagraj-kulkarni]

ok Thank you

[ipspace]

Anyway, it looks like I didn't stress enough the importance of using the (latest version of) @hellt fork of vrnetlab to build the container images, so I (hopefully) fixed the documentation: 20991a5

[nagraj-kulkarni]

Thanks for the quick responses :) for now, i pointed the default gw as suggested by Steffano and it helped. i was in such an awful situation :) will try with vrnetlab fork..Thanks again..

[ipspace]

Here's another idea to build that Vagrant box:

• Start a ubuntu:22.04 container (making sure to map the current directory or some such into it)
• Connect to the container, run the netlab / ubuntu / libvirt installation procedure
• Build the Vagrant box within the container
• Ensure the box-building artifacts are in the host directory mapped into the container (and thus accessible from your server)
• Install Vagrant box on your server with vagrant box add json-file command

[nagraj-kulkarni]

guess i am the privileged person :)

[nagraj-kulkarni]

@ipspace @ssasso on spine-leaf lab ( fabric: plugin) with csr docker image , see lldp neighbours keep changing. do you know of any known issues/caveats apart from MTU size and sub-interfaces support?

leaf-1#show lldp ne

Capability codes:
(R) Router, (B) Bridge, (T) Telephone, (C) DOCSIS Cable Device
(W) WLAN Access Point, (P) Repeater, (S) Station, (O) Other

Device ID Local Intf Hold-time Capability Port ID
spine-1.example Gi2 120 R Gi2

Total entries displayed: 1

leaf-1#show lldp ne

Capability codes:
(R) Router, (B) Bridge, (T) Telephone, (C) DOCSIS Cable Device
(W) WLAN Access Point, (P) Repeater, (S) Station, (O) Other

Device ID Local Intf Hold-time Capability Port ID
spine-2.example Gi3 120 R Gi2

Total entries displayed: 1

leaf-1#

leaf-1#show lldp ne

Capability codes:
(R) Router, (B) Bridge, (T) Telephone, (C) DOCSIS Cable Device
(W) WLAN Access Point, (P) Repeater, (S) Station, (O) Other

Device ID Local Intf Hold-time Capability Port ID
spine-1.example Gi2 120 R Gi2

Total entries displayed: 1

Topology.yaml =======>

provider: clab
defaults.devices.csr.clab.image: "vrnetlab/vr-csr:17.03.08"
plugin: [ fabric ]
defaults.device: csr

fabric.spines: 2
fabric.leafs: 4

fabric.leaf.name: leaf-{count}
fabric.spine.name: spine-{count}

groups:
leafs:
module: [ ospf, bgp ]
bgp.as: 65000
spines:
module: [ ospf ]

nodes:
host-1:
device: linux
image: ubuntu:latest
host-2:
device: linux
image: ubuntu:latest

links:

• host-1:
  leaf-1:
• host-2:
  leaf-2:

[ipspace]

A) It might be a good idea to open a new discussion for a completely unrelated issue
B) No idea. You could try CDP/LLDP debugging. Without knowing anything else, it looks like either the LLDP packets are getting lost or there's a timeout mismatch.

You could try the new packet capture capability added in 1.9.1 to figure out what's going on.

[ipspace]

Ran a few tests:

A) Contrary to what I said, you cannot do packet capture on vrnetlab containers because the outside container interface names don't match the inside names (#1380)
B) I'm experiencing the exact same behavior. As I have plenty of CPU and can see the LLDP packets on both uplinks, it's not resource related. It looks like all CSRs report the same Chassis ID MAC address with different management IP addresses (loopback interfaces) and that might confuse the LLDP process.

I also checked the serial numbers, and it looks like all vrnetlab-based CSRs have the same serial number. Have no idea how to change that or the Chassis ID, it might be based on the MAC address of the first interface (and you know what the problem is there).

However, LLDP does work correctly with the CSR VMs:

leaf_1#show lldp neighbors
Capability codes:
    (R) Router, (B) Bridge, (T) Telephone, (C) DOCSIS Cable Device
    (W) WLAN Access Point, (P) Repeater, (S) Station, (O) Other

Device ID           Local Intf     Hold-time  Capability      Port ID
spine_1.lab.local   Gi2            120        R               Gi2
spine_2.lab.local   Gi3            120        R               Gi2

Total entries displayed: 2

You might try to change the vrnetlab code to make the MAC address random (or open an issue and ask someone else to do it).

[nagraj-kulkarni]

great, Thanks a lot @ipspace for checking on this and all the details. When i checked on the CSRs that they are waiting for lldp frame, was wondering how to capture the packet and check.