Wikimedia Release Tools

Wikimedia Release Tools

Collection of data engineering DAGs to be executed by the WMF Airflow instances.

In the heart of the Section topics data pipeline lies a set of Spark jobs.

This project takes care of creating the docker image required to run the RDF Streaming Updater. The RDF Streaming Updater is a flink job that replicates updates made to wikidata and commons into blazegraph.

testing AQS services

A container registry implemented in Python which supports automatic online garbage collection of unreferenced blobs.

Repository for configuration of general purpose GitLab Cloud Runner.

The mw CLI tool (with development environment & gitlab commands)