ipverse.net UA (Ukraine) bug

[frankofno]

I just updated to the latest version of your script and started using ipverse.net since ipdeny is having issues with the ssl. So I ran the script with the following options:

/usr/local/bin/nft-geo-filter --table-family netdev --table-name SPAMMERS --interface eth0 --no-ipv6 --provider ipverse.net CN RU RO UA TR IR ID MY VN TH SG IN

I get that error message showing up:
ERROR - update_filter_set - Could not add the "ua" IPv4 blocks to the filter-v4 set in SPAMMERS
WARNING - restore_old_sets - No old sets detected. Setting the SPAMMERS table as dormant!
ERROR - show_subprocess_run_error - Failed to run: (1, ['/usr/sbin/nft', '-f', '/tmp/tmpo797kab5/tmpetgiynsf'])
ERROR - show_subprocess_run_error - Command exit status: 1

ERROR - show_subprocess_run_error - Command stdout:

ERROR - show_subprocess_run_error - Command stderr:
netlink: Error: Could not process rule: No space left on device

Although it looks like, all IPS and country blocks where added correctly. Not sure what to do with that messages.

I can not use UA (Ukraine) in the country list. If I remove UA from the block list, everything is working fine.

[palight]

Hello, i get the same error with :
nft-geo-filter -a --allow-established --provider ipverse.net CH FR SK
ERROR - update_filter_set - Could not add the "fr" IPv4 blocks to the filter-v4 set in geo-filter
WARNING - restore_old_sets - No old sets detected. Setting the geo-filter table as dormant!
ERROR - show_subprocess_run_error - Failed to run: (1, ['/usr/sbin/nft', '-f', '/tmp/tmpn873hb17/tmp_8a7kp9l'])
ERROR - show_subprocess_run_error - Command exit status: 1

ERROR - show_subprocess_run_error - Command stdout:

ERROR - show_subprocess_run_error - Command stderr:

After that my table seems ok, but people from US can stil use my server...

Any help would be appreciate.

EDIT : is it possible that's because ram is full on device ?

[rpthms]

@palight Could you try running the command with the TMPDIR environment variable? Just set the TMPDIR to some directory that's not a tmpfs directory like /tmp. You could use some random directory in your home directory

TMPDIR=/home/user/some_random_dir nft-geo-filter -a --allow-established --provider ipverse.net CH FR SK

[rpthms]

@frankofno Your issue is one with nftables unfortunately, apparently when there is an IP block overlap between multiple countries, nftables fails to add the IPs to the filter set. I'm not sure what to do here right now, I'll have to think a bit more about it.

[palight]

@rpthms
Hello, same error if I run it in tempdir=/root

ERROR - update_filter_set - Could not add the "fr" IPv4 blocks to the filter-v4 set in geo-filter
ERROR - show_subprocess_run_error - Failed to run: (1, ['/usr/sbin/nft', '-f', '/root/tmpvxbnxros/tmpuiylqj62'])
ERROR - show_subprocess_run_error - Command exit status: 1

ERROR - show_subprocess_run_error - Command stdout:

ERROR - show_subprocess_run_error - Command stderr:

Maybe i should add that i run it in an debian lxc container.

[rpthms]

@palight Could you open a new issue? Your issue is not the same one that @frankofno is facing. Also, in the new issue, could you post the debug log by running nft-geo-filter with the -vv flag:

TMPDIR=/root nft-geo-filter -vv -a --allow-established --provider ipverse.net CH FR SK

[frankofno]

is this due to the fact, that ipverse is generating "optimized blocks"? Because that never happend with ipdeny. I guess it's a sort issue? If I move the UA for example in the list, before RO, the message tells me RO is the problem. I guess @palight is experiencing the same issue with other countries.

@frankofno Your issue is one with nftables unfortunately, apparently when there is an IP block overlap between multiple countries, nftables fails to add the IPs to the filter set. I'm not sure what to do here right now, I'll have to think a bit more about it.

[palight]

Exactly the same. If i move FR before CH, I got the error for CH.

[rpthms]

I guess I might have to create separate sets for each country's IP blocks to fix this issue. That might take a while for me to get to. I can't think of any other easy to implement fixes right now.

[Reiner030]

yes, looks that this is the error I also run today with following line:

# /root/bin/nft-geo-filter --allow-established --allow DE US --verbose
INFO - add_table - Adding a inet table: geo-filter
INFO - add_chain - Adding the filter-chain in the geo-filter table
INFO - find_old_rules - Finding old filtering rules in the filter-chain of the geo-filter table
INFO - does_set_exist - Checking if the filter-v4 set exists in the geo-filter table
INFO - does_set_exist - Found set filter-v4 in geo-filter!
INFO - flush_filter_set - Flushing the filter-v4 set in the geo-filter table
INFO - set_table_as_dormant - geo-filter is dormant: True
INFO - get_ip_blocks - Downloading "de" IPv4 blocks from ipverse.net
INFO - get_ip_blocks - Building list of IPv4 blocks for de..
INFO - set_table_as_dormant - geo-filter is dormant: False
INFO - update_filter_set - Adding the "de" IPv4 blocks to the filter-v4 set in geo-filter
INFO - set_table_as_dormant - geo-filter is dormant: True
INFO - get_ip_blocks - Downloading "us" IPv4 blocks from ipverse.net
INFO - get_ip_blocks - Building list of IPv4 blocks for us..
INFO - set_table_as_dormant - geo-filter is dormant: False
INFO - update_filter_set - Adding the "us" IPv4 blocks to the filter-v4 set in geo-filter
ERROR - update_filter_set - Could not add the "us" IPv4 blocks to the filter-v4 set in geo-filter
INFO - restore_old_sets - Restoring the old sets in the geo-filter table
ERROR - show_subprocess_run_error - Failed to run: (1, ['/usr/sbin/nft', '-f', '/tmp/tmp2ftpp854/tmphpkdy_c5'])
ERROR - show_subprocess_run_error - Command exit status: 1

ERROR - show_subprocess_run_error - Command stdout:

ERROR - show_subprocess_run_error - Command stderr:
netlink: Error: Could not process rule: No buffer space available

INFO - delete_working_dir - Deleting the working directory

When I fun the command with only US country in whitelist it works:

# /root/bin/nft-geo-filter --allow-established --allow US --verbose
INFO - add_table - Adding a inet table: geo-filter
INFO - add_chain - Adding the filter-chain in the geo-filter table
INFO - find_old_rules - Finding old filtering rules in the filter-chain of the geo-filter table
INFO - does_set_exist - Checking if the filter-v4 set exists in the geo-filter table
INFO - does_set_exist - Found set filter-v4 in geo-filter!
INFO - flush_filter_set - Flushing the filter-v4 set in the geo-filter table
INFO - set_table_as_dormant - geo-filter is dormant: True
INFO - get_ip_blocks - Downloading "us" IPv4 blocks from ipverse.net
INFO - get_ip_blocks - Building list of IPv4 blocks for us..
INFO - set_table_as_dormant - geo-filter is dormant: False
INFO - update_filter_set - Adding the "us" IPv4 blocks to the filter-v4 set in geo-filter
INFO - add_filtering_rule - Adding a new filtering rule for IPv4 addresses in geo-filter's filter-chain
INFO - does_set_exist - Checking if the filter-v6 set exists in the geo-filter table
INFO - does_set_exist - Found set filter-v6 in geo-filter!
INFO - flush_filter_set - Flushing the filter-v6 set in the geo-filter table
INFO - set_table_as_dormant - geo-filter is dormant: True
INFO - get_ip_blocks - Downloading "us" IPv6 blocks from ipverse.net
INFO - get_ip_blocks - Building list of IPv6 blocks for us..
INFO - set_table_as_dormant - geo-filter is dormant: False
INFO - update_filter_set - Adding the "us" IPv6 blocks to the filter-v6 set in geo-filter
INFO - add_filtering_rule - Adding a new filtering rule for IPv6 addresses in geo-filter's filter-chain
INFO - add_allow_rules - Allow private IPv4 address ranges in geo-filter's filter-chain
INFO - add_allow_rules - Allow link local IPv6 traffic in geo-filter's filter-chain
INFO - allow_established - Adding a rule to allow packets from established connections in geo-filter's filter-chain
INFO - delete_old_rules - Deleting old filtering rules from geo-filter's filter-chain
INFO - delete_working_dir - Deleting the working directory

From netlink: Error: Could not process rule: No buffer space available it seems clear that the buffer for network rules seems not big enough for such table push which I found in issues google/nftables#103 and https://github.com/projectcalico/felix/issues/595 .
Question is if there is a workaround by pushing prepared table in multipes steps or if there is a chance to increase buffer size ?

[frankofno]

i was wandering, if this bug is fixed with the latest version? I just got new errors when using IN (India) as country to block.