Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • Airflow DAGs Airflow DAGs
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Custom issue tracker
    • Custom issue tracker
  • Merge requests 7
    • Merge requests 7
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Activity
  • Graph
  • Jobs
  • Commits
Collapse sidebar
  • repos
  • data-engineering
  • Airflow DAGsAirflow DAGs
  • Merge requests
  • !123

Draft: Instrument lineage of airflow pipelines

  • Review changes

  • Download
  • Email patches
  • Plain diff
Open Milimetric requested to merge add-datahub-lineage into main Aug 19, 2022
  • Overview 0
  • Commits 1
  • Pipelines 1
  • Changes 1

The docs below outline the setup necessary to enable an Airflow instance to talk to our Datahub instance. Once these are done, we can test this MR to make sure it works, and then review it for code style/approach.

The goal is to have lineage defined for all existing Airflow DAGs so we can set a good example going forward.

Refernce: https://datahubproject.io/docs/lineage/airflow/ https://airflow.apache.org/docs/apache-airflow/stable/lineage.html

Edited Aug 19, 2022 by Milimetric
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: add-datahub-lineage