edit_intent: add notebooks for predicting edit intent from diffs

Fabian Kaelinrequested to merge
edit_intent into main

Add notebooks that build a diff-based dataset for predicting the edit intent label (derived from the edit summary) from the revision diff, plus baseline models and exploratory analysis.

Foundation:

  • A shared library with config, the structured mwedittypes JSON parser, and diff-to-added/removed-text extractors. The edit summary is dropped on read to prevent label leakage.
  • A raw features notebook that enriches ~10.9M interpretable enwiki edits with wikitext, the parent diff, and edit-types JSON, written to research_dev.edit_intent_raw.
  • An exploratory analysis notebook: join coverage, label distribution, per-intent diff size and edit-type composition, and a leakage audit.

Three starter use cases on top of the raw features:

  • Logistic-regression intent classifier features and a baseline fit.
  • Clustering on edit-type feature vectors.
  • Clustering on embeddings of the words each revision added, using intfloat/multilingual-e5-large-instruct.

Outputs are Iceberg tables in the research_dev database.

Assisted-by: Claude Opus 4.8 Bug: T433134

Merge request reports