Skip to content

T382407 Update external instance URI for rest external task sensor

Andrew McAllister (WMDE) requested to merge T382407-wd-query-segments-dag into main

Contributor checklist

  • I have written tests for this DAG that will be merged into data-engineering/airflow-dags/tests/wmde
    • Tests ran in prior merge requests in #865 and #881
  • I have locally ran the above tests and code quality checks as outlined in the tests section of the Airflow DAGs project readme
    • Note: I'm getting an error on a non-existing search.shared directory/module in current tests
  • I have tested the jobs for this DAG in my local database using the process defined in wmde/analytics/hql/airflow_jobs/PATH_TO_JOB_TESTS
    • Task tests were ran for #865 and #881, and this change is only for the DAG
  • I have tested the included DAGs in my local database using the process outlined in TEST_AIRFLOW_DAGS.md and the test variable files provided for each DAG
    • Note that local tests aren't possible for local instances using RestExternalTaskSensor at this time
  • All Hive tables that are needed by the included DAG jobs have been created and are accessible by the analytics-wmde Airflow user
    • wmde.wd_query_segments_daily already exists

Description

  • T382407: Currently the DAG that generates wmde.wd_query_segments_daily has a malfunctioning sensor. The goal here is to see if switching it over to airflow-search.discovery.wmnet will work.

Test outputs

  • Task outputs are not needed here as the DAG is already deployed and we just want to unbreak it.
Edited by Andrew McAllister (WMDE)

Merge request reports

Loading