Add spark support for shipping conda environments and DRY http proxy setting propagation

Created by: ottomata

This helps automate shipping of conda environments to remote YARN workers when there are dependencies needed there that are not installed there by default.

This also properly sets up PYSPARK_PYTHON to use anaconda-wmf on remote YARN workers if the user chooses not to ship a conda environment of their own. anaconda-wmf is installed everywhere and is a better default for remote python than the system installed one.

Bug: T272313