BGP

I just recently made a new script which use whois to get subnets from corporations, and as I stated it is quite inefficient, so I only run it once a week. I later found out that there is a much easier, and more accurate, way to retrieve these lists, and that is tapping into BGP ¹. To avoid going into too much detail about this obviously much important protocol which makes internet go around, what I was recommended to use was bgpq4 ². After some quick testing, it presented lists not only much faster than whois, but also with higher accuracy, as more items was presented. I also do not have to abuse any whois service.

I therefore went to upgrade my firewall script with an additional function:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

def get_asn_subnets(ls: list[str], is_ipv6: bool):
  """
  From a list of ASN, get their list of subnets
  """
  ipv = '-6' if is_ipv6 else '-4'
  for asn in ls:
    cmd = ['bgpq3', ipv, '-F', '%n/%l\\n', asn]
    with subprocess.Popen(cmd,
        stdout=subprocess.PIPE, stderr=subprocess.PIPE,
        bufsize=0, universal_newlines=True) as p:
      lines = []
      for line in p.stdout:
        line = line.strip()
        if line:
          lines.append(line)
      ips = [ipaddress.ip_network(subnet) for subnet in lines]
      for ip in ipaddress.collapse_addresses(ips):
        yield ip.compressed

As one may notice, I am using bgpq3, but that was only what VyOS offered, and the changes are negligible to affect our result. I also took some time to refactor and redesign the structure of previously mentioned script, as adding this feature made it a bit clunky to use, and the redesign did not cause any big issues in general, but backwards compatibility.

In total, while VyOS now need to work harder to get some of the lists, after adding more subnets to block, it still takes only 10 seconds to run. I would expect this would increase as more subnets are added, but luckily nftables is very efficient to handle these lists.