Explore projects

Libraries for working with the Wikimedia Foundation's Event Platform.

https://wikitech.wikimedia.org/wiki/Event_Platform

PoC for consuming revision and android interaction streams

A set of Apache Spark powered tools which are used to transform data and metrics via a set of Airflow DAGs.

Java Flink application which creates a Kafka stream of Wikidata page modifications in RDF format.

Creates an artifact, a packed conda environment, to be deployed across the data engineering cluster. Currently contains Pyspark3 and JupyterHub.