Skip to content

Switch to Dumps wikitext_raw_rc2 schema.

Xcollazo requested to merge wikitext_raw_rc2 into main

In this MR we pickup the changes from repos/data-engineering/dumps/mediawiki-content-dump!18 (merged).

Additionally, we:

  • Consolidate page content change and visibility change stream consumption into one DAG, and make sure we consume page content events before visibility events. This helps for partial correctness. There are still unlikely but possible correctness issues being discussed elsewhere.
  • Start writing to a new version of wikitext_raw : wikitext_raw_rc2.

Bug: T340863

Edited by Xcollazo

Merge request reports