Skip to content

test_k8s: introduce a DAG submitting a simple Spark job to YARN in cluster mode

Brouberol requested to merge T377928 into main

This will allow us to check that the YARN integration works as expected for jobs running a spark-submit job in cluster deploy_mode (both the driver and the application run in YARN itself).

Signed-off-by: Balthazar Rouberol brouberol@wikimedia.org Bug: T377928 Depends-On: https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1087903

Merge request reports