Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • Airflow DAGs Airflow DAGs
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Custom issue tracker
    • Custom issue tracker
  • Merge requests 9
    • Merge requests 9
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Activity
  • Graph
  • Jobs
  • Commits
Collapse sidebar
  • repos
  • data-engineering
  • Airflow DAGsAirflow DAGs
  • Merge requests
  • !62

Update spark3 analytics for SparkSQLNoCLIDriver

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Joal requested to merge spark3_sql into main May 24, 2022
  • Overview 12
  • Commits 19
  • Pipelines 19
  • Changes 10

This patch hacks the analytics spark3 configuration to use the SparkSQLNoCLIDriver class instead of the default one, making it work in cluster mode. This is important until Spark3 gets installed on all hadoop workers and it can be run in client with Skein.

Bug: T306955

Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: spark3_sql