Skip to content

search: drop_dated_directories.py must run through skein

Ebernhardson requested to merge work/ebernhardson/drop-dated-skein into main

This was mistakenly using a SparkSubmitOperator to handle the task submission, but this script does not use spark. It is a simple python script that interacts with hdfs through pyarrow. Switch the task definition to a SimpleSkeinOperator to get it running.

Merge request reports