Ubuntu firewall backend

[DevTAGit]

To my limited knowledge, Ubuntu is using both iptables and nftables, where iptables rules are being translated to nftables rules using iptables-nft. Since both iptables and nftables can be used with geoip-shell, what firewall backend should be used on Ubuntu and are there any problems with using one over another?

As a side note, perhaps it would make sense to interrupt configuration process with a warning in case iptables are detected and ipset is not installed instead of just jumping to nftables? During my first installation attempt I didn't bother to check whether an ipset package was present, which resulted in nftables being used.

[friendly-bits]

To my limited knowledge, Ubuntu is using both iptables and nftables, where iptables rules are being translated to nftables rules using iptables-nft.

This sounds correct for a relatively modern Ubuntu version.

Since both iptables and nftables can be used with geoip-shell, what firewall backend should be used on Ubuntu and are there any problems with using one over another?

I can't make a comprehensive comparative writeup here (and there is enough available info online) but I'll write what I think is most important for geoip-shell.

Both iptables and nftables are essentially an interface with the kernel netfilter component. nftables is a newer implementation with a completely different rules syntax. Which is why if you want to use iptables rules on a system which has nftables, they give you the iptables-nft compatibility layer which accepts iptables-style syntax and internally translates it to nftables rules. As far as my testing with geoip-shell goes, iptables-nft is functionally indistinguishable from a relatively modern version of iptables and works 100% reliably. Probably there are some edge cases where it fails but I haven't encountered them. One major difference is that nftables supports nftables sets while iptables doesn't have a built-in set implementation, which is why the ipset package is required - the ipset utility provides this functionality. While (to my incomplete knowledge) the set members eventually live in the same memory space as far as the kernel is concerned, the user interface with the sets is different. This is significant because unfortunately, nftables sets implementation is both better and worse. It is better in the sense that nftables supports the auto-merge flag for sets. When this flag is set (and geoip-shell sets it), nftables will merge any adjacent or overlapping IP ranges. With most typical IP lists for countries, this significantly reduces the total count of IP ranges which make it into the set. So firewall will be more efficient/consume less CPU cycles when processing traffic with nftables. This may or may not matter, depending on your particular setup, hardware, traffic conditions and the selection of country codes. On the flip side, the nftables interface with sets is, unfortunately, not as good - at least for now. There are known long-standing bugs, mostly manifesting as much higher (compared to iptables+ipset) memory use spikes when loading or changing sets, and delays with listing set members which become progressively larger with higher total count of set members (across all loaded sets). The latter issue is insignificant with lower total members count (let's say a 100k-300k) and then grows exponentially to a degree where if you have 2-3M total set members, listing or modifying set members may take seconds to minutes.

As a side note, perhaps it would make sense to interrupt configuration process with a warning in case iptables are detected and ipset is not installed instead of just jumping to nftables?

The interactive setup procedure has to balance verbosity/completeness with usability and code compactness. As geoip-shell has a lot of options, walking the user through every possible configuration option would be too much both for the user and for the code base. I agree that the firewall backend is an important configuration aspect, which is why I implemented a dialog which allows the user to select one. With the current implementation, that dialog is skipped when only one firewall backend is available (and as far as geoip-shell is concerned, iptables firewall backend consists of iptables+ipset). The reason is I believe that a large majority of users don't want to bother with thinking about the firewall backend, and probably many don't have the domain knowledge about this. I think that the average user just wants to get the setup over with, so the setup procedure is optimized for that. I did implement specific messages printed at the first setup which explicitly says which firewall backend is being used, including in one of the last printed lines, in order to make this more visible. So users who have the time and the interest to think about the firewall backend get the opportunity and the reminder to do so. The configured backend is also specifically printed in the geoip-shell status output, and the README mentions support for both backends. The geoip-shell -h output lists the relevant configuration options as well. So I think anyone interested has enough clues to consider changing the firewall backend if they want to. Yes, if they decide that they want to use the other supported firewall backend (and use the command geoip-shell configure -w <nft|ipt> - reinstallation not required, except on OpenWrt) then geoip-shell will re-download and re-process the IP lists, but this is typically a fairly quick thing (several minutes at most) and only needs to be done once.

If you have a good argument why this behavior should be changed, I am ready to hear it.

[DevTAGit]

Thanks a lot for such a detailed answer.

As far as my testing with geoip-shell goes, iptables-nft is functionally indistinguishable from a relatively modern version of iptables and works 100% reliably.

Since you were conducting tests with iptables-nft, I assume there will be no problems with using iptables-nft rules of geoip-shell? It seems to make more sense to use nftables directly but there are other applications that are using iptables-nft in my setup, so it would be nice to have a consistency.

I think that the average user just wants to get the setup over with, so the setup procedure is optimized for that.

A perfectly valid point. I just thought that if someone has both iptables and nftables installed there must be a reason for that, thus the abscense of ipset might be an oversight on user's side. As you pointed, the system provides all the info about setup and reconfiguration process takes very little time, so it's indeed not a big deal.

[friendly-bits]

Since you were conducting tests with iptables-nft, I assume there will be no problems with using iptables-nft rules of geoip-shell?

I have never encountered any issues with iptables-nft - as I wrote above, it appears to be working 100% reliably in the context of geoip-shell, indistinguishable from iptables. It also integrates with the ipset utility exactly the same way as iptables does. So yes, geoip-shell can be used with iptables-nft without any issues, as far as I'm aware.

I have encountered some compatibility issues across different versions of iptables which would occasionally manifest as bugs in geoip-shell (e.g. some versions would print out rules in a slightly different way than others, which the built-in geoip-shell parser would not account for, or in one case a certain ipset version would enforce set name length limitation when other versions wouldn't), but those have been all solved at this point.

One thing to consider is that geoip-shell is a small project developed by one person in their (my) free time, so pre-release testing is limited and not always enough to reveal all bugs. Supporting 2 firewall backends obviously creates more surface area for bugs and generally nftables support tends to get more testing than iptables. So over the course of geoip-shell development, there have been more cases of iptables-related regressions than nftables-related ones. That said, when such bugs surface, they tend to get reported fairly quickly, and I subsequently fix them. The current version of geoip-shell (v0.7.8.1) has been available for a month now with only one bug report (which manifests with some specific versions of Bash), so currently geoip-shell appears to be working well with either firewall backend. When upgrading geoip-shell in the future (btw the next release is not too far away at this point), you should make some rudimentary functionality testing to make sure no bugs affecting your setup surfaced (and report them if they did).

It seems to make more sense to use nftables directly but there are other applications that are using iptables-nft in my setup, so it would be nice to have a consistency.

Under these conditions and unless the firewall is seeing a lot of traffic where performance becomes critical, my recommendation is stick with iptables. There is nothing wrong with using the older implementation when you do not need the features of the newer one. And the widely known advice is to avoid mixing iptables and nftables rules on the same machine if possible.