Skip to content

convert_to_esbulk: Overwrite the output path if it already exists

Ebernhardson requested to merge overwrite-text-output into main

This script is different from all the other ones. Everything else writes to a hive table, and will overwrite any partition that already exists. This script though had no overwrite option, which means it needed a different process from the other scripts for retrying failures.

To get everything behaving similarly use pyarrow to delete the output directory if it already exists.

Merge request reports