Skip to content
GitLab

Emit event counts, invocation time, and python process memory usage for process function

Merged Ottomata requested to merge T328925_metrics into main

Changes:

  • DataStream.process is always used via FlinkStreamManager.process, This way, metrics will always be collected, even if not emitting error events.

  • ErrorHandlingProcessFunction has been renamed to EventProcessFunction. This function handles emitting metrics and error events. If error_output_tag is None, no error events will be emitted.

  • Added an optional name parameter to StreamManager interface FlinkStreamManager uses this to name the various Flink operators, which results in much nicer metrics. We might want to consider making this parameter non optional in the future.

Bug: T328925