Skip to content

airflow-main/canary_events: lower the resources allotted to canary events

Brouberol requested to merge canary-events-resources into main

After having inspected the grafana dashboard over the entire dag run of a canary_events dag, it appears that much of the pods running the SkeinOperator tasks are given way too much resources, which has a visible impacy on the cluster as a whole.

By looking at the dag grid view, we know that it should run under ~7 minutes. We'd like to make sure the resources impact on the cluster is minimized, while keeping the dag run execution speed in that ballpark.

CPU/Memory requests/limits for the airflow-main namespace Screenshot_2025-04-15_at_15.50.58

CPU/Memory requests/limits for the whole dse-k8s-eqiadcluster Screenshot_2025-04-15_at_15.51.07

Signed-off-by: Balthazar Rouberol brouberol@wikimedia.org

Merge request reports