Skip to content

Add Spark tuning varprops to pageview_hourly DAG.

Xcollazo requested to merge fix-pageview_hourly into main

Just like with !469 (merged), we find ourselves again in a position in which we need to tune Spark settings.

This fix unbreaks the pageview_hourly DAG.

More info on opsweek email: https://lists.wikimedia.org/hyperkitty/list/data-engineering-alerts@lists.wikimedia.org/thread/AX27YJC4N4B7A5GFD6DUVR5R7IAEDVVN/

Side note: I intend to follow up with a phab task to identify all DAGs were we are missing these Spark tunings to improve opsweek.

Edited by Xcollazo

Merge request reports