Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • E Eventutilities Python
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Custom issue tracker
    • Custom issue tracker
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Activity
  • Graph
  • Jobs
  • Commits
Collapse sidebar
  • repos
  • data-engineering
  • Eventutilities Python
  • Merge requests
  • !2

Init project with basic CI and wheel publishing

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Gmodena requested to merge init-project into main Dec 08, 2022
  • Overview 20
  • Commits 16
  • Pipelines 12
  • Changes 17

This is an initial import of code developed for https://www.mediawiki.org/wiki/Platform_Engineering_Team/Event_Platform_Value_Stream/Evaluate_a_pyflink_version_of_Mediawiki_Stream_Enrichment.

It follows the python module template from workflow utils, and packages the library using setuptools. This MR introduces a basic set of functionalities that can be iteratively improved. CI provides:

  • linting (flake8)
  • running pyest
  • building and publishing a wheel.

mypy type checking is configured by not automated by default. Release management configs are provided, but not automated either at this stage.

There's some boilerplate that can be removed once we upstream to eventutilities, but that I find handy at the current stage.

Getting started

You can install eventutilities-python with test/dev dependencies with

pip install -e .[test]
pip install tox

CONTRIBUTING.md contains basic pointers for getting started. The project is managed by tox. Test and codes tyle checked can be harnessed in a local conda install or via Docker by setting the USE_DOCKER=true shell variable.

E.g.

make test

Will run pytest (tox -e py37) locally

USE_DOCKER=true make test

Will run pytest in a linux x86_64 container based atop Wikimedia python images (= the same we run in CI)

Why docker?

Docker has been provided to aid developemnt and testing on non linux / non x86 systems (pyflink won't install Apple M1 chips).

cc / @otto @tchin @lbowmaker

Edited Dec 09, 2022 by Gmodena
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: init-project