Update spark configuration
Change the global configuration for jobs to run in the production queue and to not use memory-overhead. Change per-job configuration to make jobs scale more easily by growing the number of maximum-executors available through dynamic allocation.