Skip to content
  • Xcollazo's avatar
    78d95f96
    analytics: Allow SparkSqlOperator to run Spark 3.3.2. Do Iceberg table... · 78d95f96
    Xcollazo authored
    analytics: Allow SparkSqlOperator to run Spark 3.3.2. Do Iceberg table maintenance with Spark 3.3.2.
    
    * Add documentation on using Spark 3.3.2.
    
    * Rename spark conda env.
    
    * Make SparkSqlOperator auto configured when Spark == 3.3.2.
    
    * Refactor code from inheritance to parameter approach to pickup Spark 3.3.2.
    
    * Make black and flake8 happy.
    
    * Add operator test.
    
    * Add a bit of docs.
    
    * Move to released artifacts.
    
    * Also include a Spark 3.3.2 compiled version of wmf-sparksqlclidriver.
    
    * Force skein. Get the analytics deprecated DAGs updated as well.
    
    * analytics: Intro SparkSql332Operator. Do Iceberg table maintenance with Spark 3.3.2.
    78d95f96
    analytics: Allow SparkSqlOperator to run Spark 3.3.2. Do Iceberg table...
    Xcollazo authored
    analytics: Allow SparkSqlOperator to run Spark 3.3.2. Do Iceberg table maintenance with Spark 3.3.2.
    
    * Add documentation on using Spark 3.3.2.
    
    * Rename spark conda env.
    
    * Make SparkSqlOperator auto configured when Spark == 3.3.2.
    
    * Refactor code from inheritance to parameter approach to pickup Spark 3.3.2.
    
    * Make black and flake8 happy.
    
    * Add operator test.
    
    * Add a bit of docs.
    
    * Move to released artifacts.
    
    * Also include a Spark 3.3.2 compiled version of wmf-sparksqlclidriver.
    
    * Force skein. Get the analytics deprecated DAGs updated as well.
    
    * analytics: Intro SparkSql332Operator. Do Iceberg table maintenance with Spark 3.3.2.
Loading