Add a DAG factory to cleanup old XCOMs from S3

This DAG will be useful to remove all large XCOMs put in S3, to avoid having cruft accumulate over time.

Given the similarity of all functions defined under wmf_airflow_comon.tasks, I regrouped them all under a wmf_airflow_common.tasks.s3 module.

For now, I'm only including the DAG in test_k8s and will include it in order instances if everything goes well.

Signed-off-by: Balthazar Rouberol brouberol@wikimedia.org Bug: T415777

Closes T415777

Edited by Brouberol

Merge request reports

Loading