Skip to content

search: mjolnir: safeguard mjolnir dags

Gmodena requested to merge mjolnir-workarounds into main

Adds two workarounds to prevent Mjolnir from getting stuck and blocking other DAGs in its pool.

Executes Mjolnir in a dedicated pool and adopts the wmf_airflow_common APIs for pool management.

The DAG instantiation has been updated to ensure that only one DAG and task can run in parallel in the pool.

Sets a timeout of 4 hours for the feature selection task (T383218).

Updates the unit test suite to allow Mjolnir jobs to run in non-sequential pools.

Bug: T383218

Merge request reports