Skip to content
  • Ottomata's avatar
    Support producing keyed messages · 828abb2f
    Ottomata authored
    This adds a new stream config setting 'message_key_fields' that maps
    from key field paths to event field paths.  The key object
    can then be hoisted out of event values.
    
    E.g.
    
      message_key_fields:
        key_field_a: sub.field.a
    
    Will result in a key object like
    
      { key_field_a: <value of event.sub.field.a>}
    
    Keys are serialized as JSON strings.
    
    librdkafka's default partitioner will produce null keys to random partitions, and
    non-null keys by consistently hashing the key to a specific partition.
    This will allow us to produce messages with the same key to the same parttition.
    
    Bug: T318846
    Change-Id: Iae478d7bc43107fc290bd1fc2bcbde08f42fce1c
    828abb2f