Skip to content

search: Port transfer_to_es from airflow 1

Ebernhardson requested to merge work/ebernhardson/transfer-to-es into main

Brings over transfer_to_es along with the one remaining dag it references. Primary changes are to the way variables are sourced and the switch to SparkSubmitOperator.for_virtualenv.

Additionally a ship_upload_events boolean config var was added which can be set to false while testing to allow everything to run but will not push the events that cause production to import the files.

Bug: T329881

Merge request reports