Skip to content

Move dump_to_hive jobs to spark3

Joal requested to merge spark3_dump_to_hive into main

This patch concerns two jobs:

  • commons_structured_data_dump_to_hive_weekly
  • wikidata_dump_to_hive_weekly

Patches:

  • Update dags default_args to use spark3
  • Rename files and dag_ids for consistency
  • Add minimal test
Edited by Aqu

Merge request reports