Skip to content
  • Ebernhardson's avatar
    spark3 / gitlab migration · 84d4c6b4
    Ebernhardson authored
    * Move requirements handling to conda to work with the
     data-engineering workflows for deploying python in analytics.
    * Update version number to, major version bump due
     to significant changes in how the project is built (but not
     the underlying functionality)
    * Updates pyspark from 2.1.0 to 3.1.2.
    * Keeps xgboost at 0.90 for now which limits supported python version
     to <=3.7.
    * Removed custom mypy stubs, they don't seem worth maintaining
    * Updates elasticsearch to 7.10.1, matching elasticsearch deployed
     in production and gaining library provided mypy types.
    * Minor updates to match code with mypy analysis of updated libraries
     and types. Mostly things like converting a generator to a list,
     updating ltr elasticsearch client for the new headers parameter passed
     by @query_params, and using queue.Queue generics directly.
    * Update SCM information in pom.xml, along with updating scala and
     spark dependencies for spark 3.1.2.
    Change-Id: I16c9550091e22eacccec8f2f215472b200b79966