Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • Airflow DAGs Airflow DAGs
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Custom issue tracker
    • Custom issue tracker
  • Merge requests 8
    • Merge requests 8
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Activity
  • Graph
  • Jobs
  • Commits
Collapse sidebar
  • repos
  • data-engineering
  • Airflow DAGsAirflow DAGs
  • Merge requests
  • !318

search: Port transfer_to_es from airflow 1

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Ebernhardson requested to merge work/ebernhardson/transfer-to-es into main Mar 22, 2023
  • Overview 0
  • Commits 1
  • Pipelines 4
  • Changes 13

Brings over transfer_to_es along with the one remaining dag it references. Primary changes are to the way variables are sourced and the switch to SparkSubmitOperator.for_virtualenv.

Additionally a ship_upload_events boolean config var was added which can be set to false while testing to allow everything to run but will not push the events that cause production to import the files.

Bug: T329881

Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: work/ebernhardson/transfer-to-es