Skip to content

Bump up spark config for cassandra load monthly top pageview articles

Mforns requested to merge bump-up-cassandra-load-monthly-top-articles-spark-config into main

The load_pageview_top_articles_to_cassandra task within the cassandra_load_monthly DAG has failed consecutively for the last 2 monthly runs. It seems the monthly size of the source data has become too big, and we need to increase the resources specified in Spark conf.

Merge request reports