Skip to content

Override cassandra loading SparkSQL application jar

Joal requested to merge fix_cassandra_loading_app_jar into main

The default jar currently use contains a version of the spark-cassandra-connector that takes precedence on the job classpath from the patched connector provided. The application jar is currently needed as we run the jobs with SparkSQLNoCLIDriver in cluster mode while Spark3 is not yet available to be used with Skein in client mode. The plan is to remove the 'application' parameter override as soon as the jobs can be run in client mode with Skein, without the need of the refinery-job jar.

Merge request reports