-
Xcollazo authored
analytics: Allow SparkSqlOperator to run Spark 3.3.2. Do Iceberg table maintenance with Spark 3.3.2. * Add documentation on using Spark 3.3.2. * Rename spark conda env. * Make SparkSqlOperator auto configured when Spark == 3.3.2. * Refactor code from inheritance to parameter approach to pickup Spark 3.3.2. * Make black and flake8 happy. * Add operator test. * Add a bit of docs. * Move to released artifacts. * Also include a Spark 3.3.2 compiled version of wmf-sparksqlclidriver. * Force skein. Get the analytics deprecated DAGs updated as well. * analytics: Intro SparkSql332Operator. Do Iceberg table maintenance with Spark 3.3.2.
Xcollazo authoredanalytics: Allow SparkSqlOperator to run Spark 3.3.2. Do Iceberg table maintenance with Spark 3.3.2. * Add documentation on using Spark 3.3.2. * Rename spark conda env. * Make SparkSqlOperator auto configured when Spark == 3.3.2. * Refactor code from inheritance to parameter approach to pickup Spark 3.3.2. * Make black and flake8 happy. * Add operator test. * Add a bit of docs. * Move to released artifacts. * Also include a Spark 3.3.2 compiled version of wmf-sparksqlclidriver. * Force skein. Get the analytics deprecated DAGs updated as well. * analytics: Intro SparkSql332Operator. Do Iceberg table maintenance with Spark 3.3.2.
Loading