Skip to content

Add unique devices DAGs

Mforns requested to merge unique_devices into main

This change adds 2 DAGs for the unique devices jobs being migrated from Oozie:

  • unique_devices_daily
  • unique_devices_monthly

Each DAG generates 2 dataset pipelines, one for unique devices per domain, and another one for unique devices per project family. Their pipelines are:

  1. Wait for wmf.pageview_actor partitions to be present
  2. Calculate the unique devices metrics
  3. Dump the metrics into a compressed CSV file
  4. Archive the dump in the final location
Edited by Mforns

Merge request reports