Add schema version to stream_manager.
Bug T327866
Add schema version to stream_manager source/destination targets, and to EventDataStreamFactory.
get_json_schema() now returns the correct version, instead of always
fetching latest.
Versioning
This change breaks compatibility. Clients should be updated.
Api changes
-
stream_managerrequires an explicit version for both source and destination streams. -
EventDataStreamFactoryfactory methods allow for 'latest'. - An
EventStreamdata class has been added to theflinkmodule.
EventStreamDescriptor
EventStreamDescriptor data class has been added to the flink module. The idea is to encapsulate stream name and
version in the same boilerplate object. This used internally in EventDataStreamFactory
factory methods to avoid method arguments proliferation. And EventStreamDescriptor can be instantiated by passing name/version to the constructor
as EventStreamDescriptor(name="some.stream", version="1.1.0") or by parsing a string EventStreamDescriptor.from_string("some.stream:1.1.0").
Note that while EventStream.from_string provides basic string parsing, naming and versioning validation is delegated to the
Java library.
For more details see docstring and unit tests under tests/test_flink.py.