Skip to content

Quote timestamp_format so it's parsed correctly by the command line

Mforns requested to merge fix-druid-load-operator-timestamp-format into main

Adds single quotes around the timestamp_format parameter in the druid_load_edit_hourly DAG. This parameter contains spaces in its value. The corresponding operator is run by SparkSkein, and will use the command line to call spark_submit. If left unquoted, it will be interpreted incorrectly by the command line:

--timestamp_format yyyy-MM-dd HH:mm:ss.S  # Will be interpreted as timestamp_format="yyyy-MM-dd"

So we have to enclose it within single quotes, so the command line interprets it as a single command:

--timestamp_format 'yyyy-MM-dd HH:mm:ss.S'  # Will be interpreted as timestamp_format="yyyy-MM-dd HH:mm:mm.S"
Edited by Mforns

Merge request reports