test_k8s: Take advantage of new KubernetesPodOperator template fields

Brouberolrequested to merge
T390186 into main

Now that a version of apache-airflow-providers-cncf-kubernetes with the following patches:

we can start taking advantage of the new KubernetesPodOperator capabilities:

  • spacing out the polling interval for the pod status (even more so for dumps of large wikis as they are bound to take hours or even days)
  • resolving the base container name as well as the pod name at runtime using templates

An issue crept through that provider release though: in the KubernetesPodOperator init method, we run

        self.container_logs = container_logs or self.base_container_name

However, self.container_logs does not belong to the template fields, so if base_container_name contains a template, it won't get rendered for container_logs. Until we resolve this upstream, we add container_logs to the WikimediaDumpOperator.template_fields attribute

Signed-off-by: Balthazar Rouberol brouberol@wikimedia.org Bug: T390186 Closes T390186

Merge request reports