Skip to content

Migrate pageview hourly, allow list, and dump jobs

Milimetric requested to merge migrate-pageview-jobs into main

Changes from the way we did it in oozie:

  • pageview allow list is checked daily and against projectview instead of pageview, for performance and convenience of those handling the alerts. This is, therefore, pulled out of the pageview hourly job.
  • dumps are processed per agent_type instead of all at once. We did this because we could not get Spark to order by properly when writing to a partitioned temp table the way Hive used to do in the oozie job.

Task: T324482

Merge request reports