Skip to content

Fix the environment of all non-base variants

Brouberol requested to merge T377928-fix-env into main

In the previous (non-base) image variants, the environment was overriden to {} because of how we had the YAML "inheritence" set-up: the whole runs block was re-defined, and thus the base environment mapping was erased back to its original default value {}.

brouberol@dse-k8s-worker1004:~$ sudo docker run -it docker-registry.discovery.wmnet/repos/data-engineering/airflow:... env | grep HOME
HOME=/home/airflow
AIRFLOW_HOME=/opt/airflow
HADOOP_HDFS_HOME=/usr/lib/hadoop-hdfs
HADOOP_HOME=/usr/lib/hadoop/
HADOOP_YARN_HOME=/usr/lib/hadoop-yarn
JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/jre
SPARK_HOME=/usr/local/lib/python3.9/site-packages/pyspark
brouberol@dse-k8s-worker1004:~$ sudo docker run -it docker-registry.discovery.wmnet/repos/data-engineering/airflow/analytics-wmde:... env | grep HOME
HOME=/home/analytics-wmde

Signed-off-by: Balthazar Rouberol brouberol@wikimedia.org Bug: T377928

Edited by Brouberol

Merge request reports