Explore projects
-
repos / data-engineering / Airflow DAGs
Apache License 2.0Collection of DAGs and common code to be executed by the WMF Airflow instances.
Updated -
Utilities and libraries for working with data pipelines at WMF. E.g. distributing conda envs, building and syncing artifacts, etc.
Updated -
repos / cloud / cicd / gitlab-ci
GNU General Public License v3.0 or laterWMCS repository for Gitlab CI/CD related stuff.
Updated -
Repository for configuration of general purpose GitLab Cloud Runner.
Updated -
Repository for configuration of Trusted GitLab Runner.
Updated -
Creates an artifact, a packed conda environment, to be deployed across the data engineering cluster. Currently contains Pyspark3 and JupyterHub.
Updated -
Don't leave Wikipedia articles and sections without images, here's the image suggestions data pipeline.
Friends call me ALIS and SLIS.
Updated -
toolforge-repos / wd-image-positions
GNU Affero General Public License v3.0Updated -
toolforge-repos / fourohfour
GNU General Public License v3.0 or laterA 404 Not Found (and other error page) handler for Toolforge
Updated -
-
repos / releng / Scap
GNU General Public License v3.0 onlyUpdated -
repos / sre / sextant
OtherA tool to manage template libraries for helm charts, and more...
Updated -
Summon our tricksy CI functions to build/test/publish your WMF deployable artifacts.
Updated -
repos / data-engineering / Eventutilities Python
Apache License 2.0Updated -
-
repos / cloud / toolforge / Maintain Harbor
MIT LicenseUpdated -
repos / cloud / toolforge / toolforge lima kilo
GNU General Public License v3.0 or laterToolforge Local Kubernetes setup for development
See also: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Kubernetes/lima-kilo
Updated -
repos / cloud / toolforge / Build service - builder
Apache License 2.0Toolforge Buildservice builder component code (currently tekton and boilerplate around using it)
Updated -
Recommend images for Wikipedia article sections based on equivalent section titles across Wikipedia language editions.
Updated -
Prometheus alerting rules for Toolforge infrastructure
Updated