PySpark jobs for the image suggestions data pipeline.

Wikimedia Release Tools

Collection of data engineering DAGs to be executed by the WMF Airflow instances.

In the heart of the Section topics data pipeline lies a set of Spark jobs.

This project takes care of creating the docker image required to run the RDF Streaming Updater. The RDF Streaming Updater is a flink job that replicates updates made to wikidata and commons into blazegraph.

testing AQS services

A container registry implemented in Python which supports automatic online garbage collection of unreferenced blobs.

Repository for configuration of general purpose GitLab Cloud Runner.

The mw CLI tool (with development environment & gitlab commands)

A simple script to deploy and run spark3 on WMF stat machines

A development environment for Scap3 and the Scap self-installer

WMF fork of moby/buildkit

Repository for configuration of Trusted GitLab Runner.