Skip to content

buildkitd: Handle bridge traffic as outbound

Dduvall requested to merge review/tell-istio-about-buildkit0 into main

Configure Istio via the annotation to ensure that traffic routed through the CNI bridge interface is handled as outbound traffic.

Istio normally routes packets coming into local interfaces as inbound traffic (destined for services running locally), redirecting them to the inbound handler on port 15006 (via PREROUTING -> ISTIO_INBOUND -> ISTIO_IN_REDIRECT). In our case, however, the local buildkit0 is a bridge for the build container namespace interfaces and is performing default routing functions, so incoming packets are really related to outbound traffic, and when Istio handles it as inbound, it seems to get very confused.

Adding the annotation tells Istio that everything coming into buildkit0 should be considered outbound, and rules are prepended to PREROUTING to ensure packets are redirected to the outbound handler.

Chain PREROUTING (policy ACCEPT) target prot opt in out source destination ISTIO_REDIRECT all -- buildkit0 * RETURN all -- buildkit0 * ISTIO_INBOUND tcp -- * * Chain ISTIO_REDIRECT (2 references) target prot opt in out source destination REDIRECT tcp -- * * redir ports 15001

Note that in this setup, the MASQUERADE rules that CNI introduces never come into play for tcp traffic, as everything is now proxied via Istio.

Downgrade the buildkit version until its entrypoint hack to bypass Istio iptables rules is removed.

Bug: T330433

Edited by Dduvall

Merge request reports