Skip to content

Extend skein operators from BaseOperator

Ebernhardson requested to merge work/ebernhardson/skein-from-baseoperator into main

When these operators were providing self.hook_submit to the PythonOperator they were capturing the self object at the time of initialization. This led to some odd inconsistencies. In particular, prior to rendering templates airflow takes a deepcopy of the task instance and applies the rendered templates to the copied instance of the task. When the copied instance of the task then ran the provided python_callable it was calling back to the non-templated instance of the task.

This resulted in, for example, the various tasks in yarn having the name of '{{ task_instance_key_str }}' instead of the expected name, even though the name field was marked as templatable.

Merge request reports