Skip to content

Initial effort to add row_type_info

Gmodena requested to merge add-rowtype-info into main

Bug: T324951

This MR adds support to pipelines that lean into Flink's Type system.

It provides wrappers to access eventutilities EventRowTypeInfo and emit Rows with the the correct type information.

This allows Rows generated from python pipelines to be consumed/produced from/to Kafka, using eventutilities SerDes and validation utils.

Commit 7215915d has been deployed on YARN for a long running integration test. See https://www.mediawiki.org/wiki/Platform_Engineering_Team/Event_Platform_Value_Stream/Pyflink_Enrichment_Service_Deployment

Helpers for working with type info in local pipelines are provided.

cc / @otto @tchin @milimetric @dcausse

Edited by Gmodena

Merge request reports