Created by: gmodena
This WIP adds the capability to extract a python script from the notebook, and save it as build artifact in Github. I open this PR to collect feedback / discuss possible approaches to releasing the notebook code in a format consumable in non-interactive mode.
The changes add a new py target to Makefile (make py) that runs nbconvert on the algo notebook. The resulting script can be executed on spark with the canonical spark-submit [...] algorithm.py.
I wanted to experiment with a flow that, after a successful build, publishes the resulting artifact to a generally available location.
I've been toying with CI and added make py to the github workflow. With this change we'd generate a new script at each PR / git push, that is available under the Artifacts pane of the Action UI.
Unfortunately there is no straightforward way to generate the artifact URL for remote consumption.
Next I want to explore the capabilities of our internal Gitlab (pipelines are enabled), and eventually move the codebase there.