Skip to content

Change rest external task sensor to new search.discovery uri

Andrew McAllister (WMDE) requested to merge update-rest-ets-uri into main

Contributor checklist

  • I have written tests for this DAG that will be merged into data-engineering/airflow-dags/tests/wmde
  • I have locally ran the above tests and code quality checks as outlined in the tests section of the Airflow DAGs project readme
  • I have tested the jobs for this DAG in my local database
    • Was done for a prior MR for this DAG
  • I have tested the included DAGs in my local database using the process outlined in TEST_AIRFLOW_DAGS.md and the test variable files provided for each DAG
    • Was done for a prior MR for this DAG
  • All Hive tables that are needed by the included DAG jobs have been created and are accessible by the analytics-wmde Airflow user
    • wmde.wd_query_segments_daily

Description

  • No task number: The airflow-search instance has now been migrated to Kubernetes, which means that we need to update the RestExternalTaskSensor for the wd_query_segments_daily DAG as it is directed at the processes that lead to discovery.processed_external_sparql_query on this instance.
  • Note: Artifacts/variables have not been updated, so this should not need a scap deployment.

Test outputs

  • No tests as this has been confirmed. We can do a hot fix if there's an issue with the next run tomorrow :)
Edited by Andrew McAllister (WMDE)

Merge request reports

Loading