PySpark jobs for the image suggestions data pipeline.

Collection of data engineering DAGs to be executed by the WMF Airflow instances.

Web2Cit translation server

Wikimedia Release Tools

Wikimedia Release Tools

In the heart of the Section topics data pipeline lies a set of Spark jobs.

This project takes care of creating the docker image required to run the RDF Streaming Updater. The RDF Streaming Updater is a flink job that replicates updates made to wikidata and commons into blazegraph.

testing AQS services

A container registry implemented in Python which supports automatic online garbage collection of unreferenced blobs.