test_k8s: Take advantage of new KubernetesPodOperator template fields
Now that a version of apache-airflow-providers-cncf-kubernetes
with the following patches:
we can start taking advantage of the new KubernetesPodOperator
capabilities:
- spacing out the polling interval for the pod status (even more so for dumps of large wikis as they are bound to take hours or even days)
- resolving the base container name as well as the pod name at runtime using templates
An issue crept through that provider release though: in the KubernetesPodOperator
init method, we run
self.container_logs = container_logs or self.base_container_name
However, self.container_logs
does not belong to the template fields, so if base_container_name
contains a template, it won't get rendered for container_logs
. Until we resolve this upstream, we add container_logs
to the WikimediaDumpOperator.template_fields
attribute
Signed-off-by: Balthazar Rouberol brouberol@wikimedia.org Bug: T390186 Closes T390186