Add schema version to stream_manager.
Bug T327866
Add schema version to stream_manager source/destination targets, and to EventDataStreamFactory.
get_json_schema() now returns the correct version, instead of always
fetching latest
.
Versioning
This change breaks compatibility. Clients should be updated.
Api changes
-
stream_manager
requires an explicit version for both source and destination streams. -
EventDataStreamFactory
factory methods allow for 'latest'. - An
EventStream
data class has been added to theflink
module.
EventStreamDescriptor
EventStreamDescriptor
data class has been added to the flink
module. The idea is to encapsulate stream name and
version in the same boilerplate object. This used internally in EventDataStreamFactory
factory methods to avoid method arguments proliferation. And EventStreamDescriptor
can be instantiated by passing name/version to the constructor
as EventStreamDescriptor(name="some.stream", version="1.1.0")
or by parsing a string EventStreamDescriptor.from_string("some.stream:1.1.0")
.
Note that while EventStream.from_string
provides basic string parsing, naming and versioning validation is delegated to the
Java library.
For more details see docstring and unit tests under tests/test_flink.py
.