Skip to content

Accept skein ApplicationSpec as dict

Ebernhardson requested to merge work/ebernhardson/popularity-score-skein into main

Changes SkeinOperator to accept a dict in the skein_application_spec argument. This allows airflow templating to apply to the dict which is then transformed into an ApplicationSpec when the operator is run. The functionality is then used to template the threshold fetching logic in search's ores dags.

While reviewing the resulting skein spec it was found that the venv handling was incorrect. Task has additionally been adjusted to source the correct venv and invoke the script from within it.

Edited by Ebernhardson

Merge request reports