I've installed Openstack Train via kolla-ansible (requires docker since the services are in containers) on 3 nodes. All 3 nodes are compute nodes and the one doubles as a controller/networking node. The 2 compute nodes run only the Openvswitch agent. Docker requires the kernel setting net.bridge.bridge-nf-call-arptables=1 (sysctl.conf) to function correctly. However, on the compute-only nodes, this breaks TCP/IP IPv4 communication with VMs on these nodes. ARP works without issue. While I have not tested IPv6, this also appears to work.

For example, creating a new instance on a compute-only node using an Ubuntu Focal cloud image leaves the Ethernet port, typically ens3, unconfigured for IPV4. ICMPv4 pings also fail on VMs on the compute-only node to for example, a Neutron router interface. Setting net.bridge.bridge-nf-call-arptables=0 resolves these issues but breaks docker and Openstack functionality, at least for services running on a Magnum cluster. I did not notice this issue on Openstack Rocky (upgrade to Stein and then immediately to Train) especially on the creation of a new VM.

As a work around I added iptables rules for the compute-only nodes per the following:

brctl show |grep 8000 |cut -f 1 |xargs -I % sudo sh -c 'if [ '%' != 'docker0' ]; then iptables -A FORWARD -i % -o % -j ACCEPT; fi;'

While this appears to resolve the issues, there may be unintended consequences that I'm unaware of. It will also require subsequent executions, or at least a script/service that runs via cron or systemd that adds new bridges, when a new bridge is added via the creation of a new VM, etc. Can anyone comment on if the above iptables rules will create other issues and if there is a better solution to this problem, e.g., via an Openstack configuration setting?