Inventory of HP Procurve switches totally useless

[marcogaio]

I'm using FI agend 2.4 on debian stretch, GLPI 9.3.2.

Inventory of HP Procurve/Aruba Network switches is totally useless: inventory is correct for a single switches per network, but if there's more then a switch, for every 'trunk' an hub in root entity get created with all the devices connected within. See the image.

If i schedule FI, at every run devices moved to that 'hub' like a 'blob' that eat all my devices.

Thanks.

[ddurieux]

You have right the LLDP activated on ALL the switches?

[marcogaio]

Mmmmmhhhh... i've still some old switces lying around (non-procurve ones) here. I've another site with only procurve switches.

Seems that LLDP is enabled by default on procurve switches:
`takeru# show lldp config

LLDP Global Configuration

LLDP Enabled [Yes] : Yes
LLDP Transmit Interval [30] : 30
LLDP Hold time Multiplier [4] : 4
LLDP Reinit Interval [2] : 2
LLDP Notification Interval [5] : 5
LLDP Fast Start Count [5] : 5

LLDP Port Configuration

Port | AdminStatus NotificationEnabled Med Topology Trap Enabled
----- + ----------- ------------------- -------------------------
1 | Tx_Rx False False
2 | Tx_Rx False False
[...]
48 | Tx_Rx False False
OOBM | Tx_Rx False False `

[ddurieux]

We can't link a problem with seems. Can you check all your switches to be sure LLDP activated ?
If you have host manage virtualmachines (esx hosts), perhaps a good idea to activate LLDP on the host too

[marcogaio]

I've chacked the site with all procurve switches. Same behaviour.

We can't link a problem with seems. Can you check all your switches to be sure LLDP activated ?
I will try... give me some time...

If you have host manage virtualmachines (esx hosts), perhaps a good idea to activate LLDP on the host too
I'm using proxmox. I've googled a bit, but seems there's no LLDP specific configuration for proxmox...

Also, sorry: can you explain how LLDP can 'mangle' all the SNMP inventory stuff? Eg:

• LLDP-enabled switches gather information from non-LLDP enabled devices, and 'pack' them in the fake 'hub' you can see..
• non LLDP switches return no data, fusion try to 'interfere' with available ones and make mistakes
• ...

If needed, i can provide of course raw SNMP walk queries and fusioninventory XML.

[g-bougard]

@marcogaio Just as a quick comment, you said you're using agent 2.4. You should upgrade it to 2.4.2 as some fixes against lldp/cdp support has been merged.

[marcogaio]

@marcogaio Just as a quick comment, you said you're using agent 2.4. You should upgrade it to 2.4.2 as some fixes against lldp/cdp support has been merged.

I'm using debian packages, that seems are still on 2.4 (debian changelog say Rebased sources on 2.4.1-dev / 00bb19f upstream github commit).

[marcogaio]

We can't link a problem with seems.

Ok, switches checked. ALL have LLDP enabled (seems by default on HP Procurve switches).

[g-bougard]

Hi @marcogaio
as you can see in the 2.4.2 news release, that latest version still include few LLDP support enhancements.
You should upgrade it: download at least the 2 following packages from Debian Sid and install them manually with dpkg command:
fusioninventory-agent_2.4.2-1_all.deb
fusioninventory-agent-task-network_2.4.2-1_all.deb

[marcogaio]

OK, debian packages downloaded from sid, installed.

Run the task. Nothing changed. Still there's 'hub' on root entities that ''eats'' all devices...

I've also revised my import rules, and i've disabled all automatic import of new mac address, and i've enabled the rule that disable import for new devices.

Still, all these MAC addresses are added, while on 'unmanaged device' FI section, where i cannot determine what rule applied for the import, right?

[dsiens]

Hello, try this fix on the inventory agent, fix bad trunk port inventory :

File : Hardware.pm ,line close to 1500, add the bold line below to the quoted code here :

sub _setAggregatePorts {
my (%params) = @_;

my $ports = $params{ports};
my $logger = $params{logger};

my $lacp_info = _getLACPInfo(%params);
if ($lacp_info) {
foreach my $interface_id (keys %$lacp_info) {
# safety check
if (!$ports->{$interface_id}) {
$logger->warning(
"unknown interface $interface_id in LACP info, ignoring"
) if $logger;
next;
}
$ports->{$interface_id}->{AGGREGATE}->{PORT} = $lacp_info->{$interface_id};

$ports->{$interface_id}->{TRUNK} = "1";

[abmteam]

Hi.
Try to add the following code at plugins/fusioninventory/inc/inventorynetworkequipmentlib.class.php file.

if (strpos($pfNetworkPort->getValue('ifdescr'),'Aggregazione') !== false) {
return;
}

function importConnectionMac()
...
      // ADD BEFORE if ($pfNetworkPort->getValue('trunk') != '1') { 

      if (strpos($pfNetworkPort->getValue('ifdescr'),'Aggregazione') !== false) {
          return;
      }

      //

      if ($pfNetworkPort->getValue('trunk') != '1') {
         if ($count == '2') {
            // detect if phone IP is one of the 2 devices
            $phonecase = 0;
            $macNotPhone_id = 0;
            $macNotPhone = '';
            $phonePort_id = 0;
 ....

[Stoatwblr]

This is the same problem I've been nagging you about forever.

Fusion makes a bunch of fatal assumptions based on "what cisco does" - which anything derived from Comware doesn't

You MUST take the long way around and match individual indexes for lldp/qbridge/vlans back to physical ports and not assume that the port indexes all match up. There is ZERO requirement for them to do so in the RFCs, nor is there any requirement for them to start at zero OR be contiguous. (ie, "holes" in the numbering are perfectly normal)

There is also no requirement for these to be consistent between switch reboots - they usually are but some don't and virtually all stackable or chassis switches have a "reset indexes" command somewhere.

Doing this will solve about half the bug reports relating to switches, eliminate almost all of the need to write special modules for every model of switch and take care of a good chunk of the wishlist.

At the moment Fusioninventory's networkinventory module is fatally flawed and this needs urgent attention as it badly affects GLPI's credibility.

[marcogaio]

Hello, try this fix on the inventory agent, fix bad trunk port inventory :
File : Hardware.pm ,line close to 1500, add the bold line below to the quoted code here :
$ports->{$interface_id}->{TRUNK} = "1";

OK, with this 1-liner patch situation seems very very better. Now i've not anymore 'hub' unmanaged device that 'eats up' all my devices. Cool! Thanks!

Still i'm not understood if this patch can be general useful, or is an 'hack' to make procurve switches work...

[marcogaio]

Hi.
Try to add the following code at plugins/fusioninventory/inc/inventorynetworkequipmentlib.class.php file.
if (strpos($pfNetworkPort->getValue('ifdescr'),'Aggregazione') !== false) { return; }

AFAI understood perl and php, this code do, GLPI side, what @criensmm patch do FI agent side. So, i've not applied.

[marcogaio]

This is the same problem I've been nagging you about forever.

Seems you know very well this field. Me, no. ;(

At the moment Fusioninventory's networkinventory module is fatally flawed and this needs urgent attention as it badly affects GLPI's credibility.

As just stated in previous bugreports (on the old ticket/code management system), i can offer raw SNMP and XML data to developers, if needed.

Thanks.

[Stoatwblr]

It won't help much.

The core issue is that FI-agent's parsing logic assumes that the if-indexes are canonical across everything else, but the problem is that they're not (in fact they're not even canonical across all all cisco ranges, not even across all Catalyst models).

The correct way to proceed is to assume that none of the indexes have any relationships to each other and then to work backwards in order to link them back to ports, which requires two passes on the snmp returned data.

This is the only safe way to get lldp/cdp, mac, vlan, qbridge, spanning tree, power, framesize, trunking and any other information to map to the correct port on the correct shelf.

Assumption is the mother of all screwups - and the assumption that's hardcoded into FI-netinventory (and the perl snmp libraries used) is that everyone does it the same way that Cisco did it in one particular model of switch. 3com did it differently in Comware, HP did it differently in their older Procurves , Bay did it a different way still, etc etc.

The result is that the only way that FI can cope with variations is to write explicit modules for each permutation of the changes. Taking the "longer" path may result in "slightly longer" processing times but the results are more consistent and that means far fewer customisations are needed (it comes down to looking at manufacturer-specific MIBs for particular returns where they don't provide generic ones (eg, temp or fan speed), or where the generics don't provide enough information, but not needing lots of manufacturer-specific coding routines)

I believe a better way forward is probably to adapt output from netdisco (which uses the "other" set of perl snmp libraries and has a far more active set of developers working on those) to produce FI-compatible output for glpi. The advantage of that is that many switches have "fragile" snmp stacks and doing it this way reduces polling which means lower loading on them (it takes around 20 minutes for FI to interrogate each of my 5-stack Huawei S5700-SI stacks. Netdisco polls them in 3-4 minutes, Cacti/mrtg/rrd gets counters in 5-6 minutes. It doesn't make sense to have all three programs making overlapping queries)

I've been pushing the FI guys to fix switch polling for a couple of years without much joy. They have other priorities (which is understandable) - and I believe adapting an existing well-supported network mapper to feed FI/GLPI would solve this problem in a much better way, reducing loads on switches and, leaving team FI free to concentrate on printers and computer inventories.

[Stoatwblr]

FWIW even with the suggested patch I get exactly the same problem still occuring on my Huawei switches thanks to the indexing mismatches resulting in this kind of error:

[debug] invalid interface ID 57 while setting vlans, aborting
[debug] invalid interface ID 57 while setting trunk flag, aborting

Which means that it never actually gets to the point of working out that the "hub" is bogus.

Attempting to make a printout of the results of the fusioninventory view at glpi/front/networkequipment.form.php would be 188 pages long - a 2-shelf 128 port core switch somehow ends up with over 1700 ports detected.

A single shelf 52-port edge switch shows 285 ports

A 3 shelf 96 port edge switch shows 572 ports.

Worse than all of that, FI has been creating and deleting so many bogus ports that glpi thinks that it's counted in excess of 11 MILLION network ports in the last couple of years, which causes database bloat and load to match. We actually have about 3000.

[g-bougard]

Hi @Stoatwblr
I think I understand your concern about if-index assumed to be canonical. I'll try to work on that subject in a month or 2 max. I hope you'll be available at that time to keep me on the right road.

[Stoatwblr]

In the meantime as we have discussed, I have written a Netdisco_2_FI4GLPI XML translator - but there are fundamental shortcomings in the XML spec (and FI4G xml import odules) we need to discuss (no way to announce if a vlan is tagged or untagged, no resolved DNS names (which are very handy) and if we know an adjacancy is a wap/phone/switch we should tell FI4G instead of making it guess.)

See #641

If anyone wants my Netdisco_2_glpi extractor then feel free to ask. It's ugly but functional and I even used what I learned from writing it to kludge up XML that can import my (non-SNMP pollable) WAP data - it commits the cardinal sin of making direct database queries but it can (and should!) be improved - it takes about 17 seconds to generate each 5-switch stack's XML at the moment and I'm sure that can be less than 5 (If these sound slow, the Fusion poller was taking 25 MINUTES to interrogate these switches and affecting performance. The other reason for wanting to derive input from Netdisco was to reduce the number of queries being sent to the switches.)

G-bougard, please don't take this the wrong way: There are hundreds (if not thousands) of devs working on various aspects of Netdisco - many of them are SNMP authors and/or fulltime paid employees of various equipment makers - and only you working fulltime on Fusion's netinventory poller. You yourself admit you have too much work on your hands.

It is far more sensible to make use of their already-existing work than to load you up with more tasks than you can complete (ie: working smarter, not working harder). By doing things this way, the MIB interfacing is already done, changes are accomodated without you needing to do anything and there's a stable export/import API already created, which means that you can concentrate on other parts of Fusioninventory..

Most of us don't care about SNMP::Info vs Info::SNMP holy wars. We just want accurate results - and with most of the SNMP RFC authors in the Info::SNMP camp, a holy war isn't sensible anyway. :)